Google Cloud Platform

Google Cloud Platform

Application and Data / Application Hosting / Cloud Hosting
Needs advice
on
Amazon EFSAmazon EFS
and
Amazon S3Amazon S3

We are creating a Document Management micro service on Google Cloud Platform and we need to choose the best Document Storage option considering all its Pros and Cons.

READ MORE
2 upvotes·521 views
Replies (2)

Since you are already on GCP, why not use their storage service for that? Another option would be to build your cluster with GCP computing instances and MinIO (https://min.io/) or to go with Backblaze (https://www.backblaze.com/). In both cases, you have an S3-compatible API; just with MinIO, you have more things available, like custom policies, hooks, and various integrations. And it is fast. Like very fast.

READ MORE
5 upvotes·59 views
DevOps | Senior Developer ·

Really depends on your use case, requirements for access, and level of development effort available for the implementation. EFS is going to be the more turnkey solution with your typical tree based folder/file system within logical volumes. In all my years though, I've never implemented it. Always S3. So rather than to speak about what I don't really know, I'll tell you what I know about S3.

S3 is object based storage. Meaning everything is essentially flat, but you can simulate the usual structure you are used to with virtual folders. It matters not what the contents are and nothing requires even a file extension. Policy contol of files and folders is extremely granular...which is a big plus. Everything should be private and then you manage roles and permissions for access control...or you can use IAM. You can obfuscate everything in your DB in order to put a file name and extention back to an object when you want to serve it up to a user. This further protects your objects even if someone gained bucket access. You can and should make "signed request" links to all files...meaning only for a small window is a file accessible to an intended users. S3 is also infinite. Meaning there is no limit to how much you can persist. While EFS is full scalable in both performance and size it also comes at a cost of elastic compute and storage. In S3, there are also lots of different storage classes meaning standard (hot - accessed frequently) all the way to Glacier (ice cold - archived, basically not accessed). You can also replicate objects to other regions for DR and redudency easily. EFS can be linked using VPC peering as well. S3 also has many useful features like lifecycle management, versioning, and many more. Virus scanning using tags and a Lambda is also an implementation to keep your files safe from harm. S3 also has advanced querying capabilities. Multipart uploads are an essential feature for massive objects (which also basically have no limit). You can chunck the file and send it in pieces for either speed or manageability. S3 can also be used with Glue and Athena to take raw data, assemble it, and then query it with common SQL language. You can also do a similar thing to perform analytics in a AWS Lake.

This is I'm sure not comprehensive, but hopefully helps. S3 will take work to build around, but will give you every opportunity you will ever want. EFS could be a drop in solution that is fast, friendly, and familiar but more costly and with less control.

READ MORE
4 upvotes·13 views
Associate Java Developer at txtsol·
Needs advice
on
JavaJava
and
Spring BootSpring Boot

I want to make application like Zomato, #Foodpanda.

Which stack is best for this? As I have expertise in Java and Angular. What is the best stack you will recommend?

Web Micro-service / Mono? Angular / React? Amazon Web Services (AWS) / Google Cloud Platform? DB : SQL or No SQL

Mob Cross-platform: React Native / Flutter

Note: We are a team of 5. what languages do you recommend if I go with microservices?

Thanks

READ MORE
8 upvotes·22.7K views
Replies (4)
Vivid Kreations - Architects and Interior Designers·

Considering your expertise in Java and Angular, I recommend a Microservices architecture using Java Spring Boot for the backend. Angular can be utilized for the frontend. AWS is a robust choice for cloud services, and for the database, a combination of SQL (like PostgreSQL) for structured data and NoSQL (like MongoDB) for flexibility in handling unstructured data. For mobile app development, React Native offers a balance of efficiency and cross-platform capability. With a team of 5, this stack ensures scalability, maintainability, and a streamlined development process for your food delivery application.

READ MORE
6 upvotes·1 comment·18.8K views
Peace MICHAELS
Peace MICHAELS
·
February 6th 2024 at 1:18PM

The best stack is the one you know today. In today's world you can build anything with most stacks. When you are starting out a project, the choice between micro-services and monolith should not be given a second thought. START WITH A MONOLITH!

·
Reply

I think you should use React or svelte and for the backend use Next.js or sveltkit and as for a database I would think you should use mongo DB and if you want to make it cross platform use flutter or React native it would be easy to use both of these if you know React they are very similar to each other and as for Next.js it is a full stack framework which uses React on the front end and nodejs at the back end good luck making the application.

READ MORE
2 upvotes·16.1K views
View all (4)
Java Software Developer at ViewNext·

Hello everyone, I would like to start using a cloud service to host my projects, which are web applications. If anyone has enough experience with Microsoft Azure, Amazon Web Services (AWS) or Google Cloud Platform, I would like to know which of these is most recommended to use, depending on the features they have or how used they are. Thank you so much.

READ MORE
8 upvotes·12.9K views
Replies (2)

Azure is only ideal if you already use Microsoft products (ASP.net, C#, etc). AWS offers a wide range of services and you get better support ( both at community and enterprise levels). It's not as easy getting help on an issue with GCP as it is with AWS.

READ MORE
7 upvotes·1 comment·7.3K views
Rik Lempens
Rik Lempens
·
January 23rd 2024 at 6:51PM

We host our SAAS solution on GCP since 2013 without any issues. The few times we needed support the GCP team was most helpful.

·
Reply
Backend Software Developer ·

AWS is a solid choice. I'd say it's the most popular, meaning there's a wealth of help and how-to guides available. This is crucial when you're just starting in this area. In terms of services, AWS offers a comprehensive range that meets all your needs, especially for beginners. You can also take advantage of the free-tier option for some services, making it easier on your wallet while you're learning the ropes.

READ MORE
5 upvotes·376 views
Needs advice
on
FlutterFlutterMongoDBMongoDB
and
Node.jsNode.js

My days of using Firebase are over! I want to move to something scalable and possibly less cheap. In the past seven days I have done my research on what type of DB best fits my needs, and have chosen to go with the nonrelational DB; MongoDB. Although I understand it, I need help understanding how to set up the architecture. I have the client app (Flutter/ Dart) that would make HTTP requests to the web server (node/express), and from there the webserver would query data from MongoDB.

How should I go about hosting the web server and MongoDb; do they have to be hosted together (this is where a lot of my confusion is)? Based on the research I've done, it seems like the standard practice would be to host on a VM provided by services such as Amazon Web Services, Google Cloud Platform, Microsoft Azure, etc. If there are better ways, such as possibly self-hosting (more responsibility), should I? Anyways, I just want to confirm with a community (you guys) to make sure I do this right, all input is highly appreciated.

READ MORE
8 upvotes·198.6K views
Replies (3)
CTO at Cloudonix LTD.·

Firstly, there's nothing wrong with Firebase for scalability, not can I see anything wrong with "cheap" - unless you expect to need the more complex tools that MongoDB offers (such as Map/Reduce, GridFS and such), I don't think you would want to pay more to get the same capabilities. That being said, there are advantages to move to an open-source code base that you have the option of hosting yourself - preventing vendor lock-in is a legitimate requirement.

Now, as for hosting MongoDB: there are a lot of providers that will host MongoDB (or compatible, see AWS DocumentDB) for you - AWS, Linode or Digital Ocean all offer managed database as a service, so you don't have to mess with VMs and installing and maintaining your own instance of the database server - they are often also cheaper than just running a VM. There's no need to co-locate the database near the application - all of those managed MongoDB services offer great connectivity so unless millisecond latency is critical for your application, any will do.

That being said, your best bet for starting to work with MongoDB is probably MongoDB's own Atlas service - it is a managed service provider that allows you to select in which cloud hosting provider to co-locate a managed MongoDB instance - they support AWS, Azure, GCP and others. They always have the latest and greatest MongoDB version (they make it themselves) and they even have a free tier for starting development on the cheap.

READ MORE
8 upvotes·40.8K views
Software Development Engineer ·

you can also use mongodb atlas, it is a dbaas, it also has shared clusters which are essentially free, good to start with. also are you planning to use an ORM( prisma or mongoose) or raw mongodb driver

READ MORE
3 upvotes·40.5K views
View all (3)
Backend Developer at Ficachi y Asociados S.C.·
Needs advice
on
Cloud FirestoreCloud Firestore
and
MongoDB AtlasMongoDB Atlas

Good day, everyone!

We recently got the request to add a feature to our systems: Forms should be saved to be continued later given the following cases:

If the connection to the server is lost, if the user closes their browser, if the user changes networks, if the user didn't finish the form and decides to take a week off and a mix of them all.

One of the best solutions we've thought so far is to save the data in the forms in a remote database every so minutes directly from the client.

A document-oriented database seems the best bet, but which database to use is the question. Since we already use Google Cloud Platform, Cloud Firestore is a "safe" option right now, but we are looking at every option we can find. The "caveat" of our use case is that we need more writes than reads, but writes are usually more expensive, and our biggest constraint is budget.

That's why I am seeking advice. What other options do we have?

In case you're wondering, we seek to save the forms bypassing the backend and our own system resources entirely, so a dedicated, remote database or Local Storage are our best options, but Local Storage is a bit controversial choice within the team, so we will explore that options later on.

Any advice or experience in this matter is highly appreciated!

READ MORE
4 upvotes·12.3K views
Replies (3)
CEO & Co-Founder at Dechea·
Recommends
on
ClientDB

You can give ClientDB a try. It simplifies storing data local, and it takes care about syncing it regularly to your remote data storage.

Advantages:

  • It's cheap, because you don't have to send a write request to your database every moment. (And it's open source)
  • If the user looses internet, it will sync up simply again at the moment he has internet again
  • You can implement an instant experience, because you don't have to wait for database feedback e.g. saving data (Not so relevant for your Use case, but maybe overall interesting to know.)

Downside:

  • You have an additional layer, that you have to maintain.
READ MORE
5 upvotes·1 comment·1.5K views
Lukas Sanger
Lukas Sanger
·
December 29th 2022 at 10:09AM

Thanks for the feedback. Considered this system for saving data

·
Reply
Software Engineer at Lumos·
Recommends
on
MongoDB Atlas

I don’t have time to go into a deeper analysis, but consider that with Cloud Firestore projects that I’ve had in the past with heavy write load, my cost was not on the writes: it was in the storage. So maybe consider that in our cost analysis - write over time + storage cost over time. Storage costs tend to be exponential and never go down unless you manually clean up your database. With Cloud Firestore this is harder than with MongoDB atlas, that has TTL for documents. By the way, I’m basing myself over an experience with Firestore from about two years ago, you might wanna check what’s the situation nowadays when it comes to archiving / deleting old data and storage costs. And last but no least, consider both Firestore and DataStore mode in your analysis. I thought there was no advantage for my use case but DataStore mode was actually more suitable at the time for a huge number of writes and deletes.

Also be mindful of performance during writes. I’ve had a bunch of performance issues with FireStore due to heavy write load and had to to batch writes. I would personally chose MongoDB Atlas as I trust MongoDB a tad more as a product, company, and always have the chance to spin up my own Mongo if costs get too high in a cloud platform.

READ MORE
5 upvotes·2 comments·4.2K views
Fernando Barcenas
Fernando Barcenas
·
August 1st 2022 at 4:26PM

Thanks! I truly didn't think about the storage cost. We do plan to deleting records with no activity in a week or so and whenever the form if fulfilled, likely on non-working hours. That being said, I will add the storage cost measure into consideration nevertheless. Is something that will add up, whether we clean the DB or not.

·
Reply
Bianca Rosa
Bianca Rosa
·
August 2nd 2022 at 5:01PM

Storage cost was a huge deal for me there - but I was in scale of 8MM events per day, so it truly depends on the size of your data - but for this size of data that can be deleted - TTL for documents is a must unless you wanna write your own batch jobs to delete data 💀

·
Reply
View all (3)
Needs advice
on
AppDynamicsAppDynamics
and
DatadogDatadog

We are evaluating an APM tool and would like to select between AppDynamics or Datadog. Our applications are largely hosted on Microsoft Azure but we would keep the option to move to AWS or Google Cloud Platform in the future.

In addition to core Azure services, we will be hosting other components - including MongoDB, Keycloak, PagerDuty, etc. Our applications are largely C# and React-based using frontend for Backend patterns and Azure API gateway. In addition, there are close to 50+ external services integrated using both REST and SOAP.

READ MORE
4 upvotes·1M views
Development manager ·
Needs advice
on
Google Cloud PlatformGoogle Cloud Platform
and
awsaws

Hi

We have 2 web applications running on PHP and Laravel and backend MySQL. Currently, each server is running on a dedicated server. We want to switch to a cloud infrastructure. Looking into either Google Cloud Platform or Amazon AWS.

Which one is better? Google cloud comes out to be a couple of hundred dollars cheaper than AWS.

Need advice on which one we should go with?

Thanks

READ MORE
5 upvotes·14.4K views
Replies (1)
Development manager ·

Any advise please?

READ MORE
4 upvotes·6 comments·10.1K views
Maggie Joseph
Maggie Joseph
·
June 13th 2022 at 7:43AM

Google Cloud Platform is better than

·
Reply
Juan Carlos Marin
Juan Carlos Marin
·
June 11th 2022 at 5:55PM

AWS would be better, as for cheaper it depends on how you do things and what are the specific requirements of your application. If, as part of your reconfiguration, you move to a container structure like using Kubernetes instead of the age old Virtual Machine way of doing things, then AWS is not only cheaper but also has the largest amount of options that deliver on value. If you get your infrastructure on one year commitments instead of on-demand, AWS is a cheaper version than Google also. The only scenario where I would choose Google, is if I had a very simple application with low traffic. When it matters AWS defeats Google and Azure every time. It is a steep curve to learn and the interfaces can be cumbersome, but making things easier doesn't add to better.

·
Reply
Sumer Singh
Sumer Singh
·
June 15th 2022 at 4:36AM

Assuming you have compared the prices.

For DB you can use any cloud managed DB for high reliability and low maintenance on your side.

Your use case is very simple and you can use pretty much any cloud.

You should think what kind of service you will use like Serverless functions/ Managed Kubernetes Service/ VM etc. But I can recommend Azure App Service (kinda service ) where you just dump your code and your application is up and running on cloud. You can fine tune some settings but its very easy to be up and running on cloud.

·
Reply
Gonçalo Rodrigues
Gonçalo Rodrigues
·
June 9th 2022 at 4:35PM

For now I would recommend GCP. If you don't wan to make the choice, you can also be cloud agnostic using Multy (https://multy.dev/)

·
Reply
Ben Baert
Ben Baert
·
June 11th 2022 at 3:42PM

Either will be fine for your needs. GCP is a little bit easier (in my opinion) and often cheaper (as you pointed out already),, AWS has a Serverless SQL database offering (Aurora) which is quite convenient. So there are some minor tradeoffs but you can't really go wrong with either.

·
Reply

Hello Everyone, My motive is to build an MLOps pipeline that is 100% independent from Cloud services like AWS, Google Cloud Platform, and Microsoft Azure. I have a project for a client in a production factory and would like to build a Camera based Object Tracking ML service for them. I want to build this pipeline in my own server or (on-premise computer). I am really confused about what stacks I should use. I keep ending up with a Cloud component based solution. It would be great to get some advice on what are the components that I can use and preferably open source.

READ MORE
5 upvotes·11.8K views
Software Engineer at Servian·
Needs advice
on
PrismaPrismaSequelizeSequelize
and
TypeORMTypeORM

Hello folks, we are making a company internal google chatbot system hosted on Google Cloud Platform. The backend is being written in ExpressJS + TypeScript, we are planning to use Cloud SQL to store basic entities like Users, Spaces, Connections(use-case specific).

The question I have is which tool is recommended for this basic use case where we want to query some data and perform CRUD operations. Also, how we should be handling the DB version control & migrations?

READ MORE
5 upvotes·8.6K views
Replies (1)
Software Consultant at CODIUM·
Recommends
on
Prisma

From my past experience, I've used TypeORM and Prisma before and found that Prisma is doing great with CRUD operation, you won't need to code much to save and read the data. And the benefit of Prisma is a single entity definition will do it all. You don't need to define a type, you don't need to define a DTO. Prisma also do t the migration as well. The (little) downsides I found for Prisma were that we need to define every entity in a "single" file, we also cannot do many operations with the queried entity since it's just an object with type, you cannot manipulate it much.

In contrast with Prisma, TypeORM allows you more flexible entity definition and more advanced SQL query (TypeORM has a built-in query builder to build a custom SQL and execute with the database, Prisma has only a "raw" function to execute raw SQL.).

In conclusion, if your application is basically a CRUD operation, with no (or few) complex SQL queries, simple and few entities, I'd recommend Prisma. Its syntax is beautiful, easy-to-use, and will make you develop faster.

READ MORE
3 upvotes·2.9K views

I am new to the cloud platforms. I am trying to figure out which Google Cloud product would be the best fit for my needs. Here are the requirements I am trying to meet:

  1. Store data in Google Cloud Platform.
  2. Support potentially thousands of concurrent users running analytical queries.
  3. The data volumes would be moderate (as far as Google is concerned) - we are talking 10's of GB up to potentially 100s of GB.
  4. The data needs to be updated once a day (but ideally a few times a day).
  5. Perhaps my biggest goal is the following: I am trying to avoid having to build a traditional data warehouse with star-schema and ETL process. I have experience with these projects, and they end up being very costly, error-prone, hard to troubleshoot, and rigid (hard to deploy changes). Instead, I am hoping to just copy my OLTP tables to GCP and query them from there, relying on GCP horsepower. I appreciate any feedback.
READ MORE
4 upvotes·7.2K views
Replies (2)

I won’t recommend any of those tools if you plan to run thousand of analytical queries each day.

In my opinion you would better use something like Druid or Clickhouse than using bigtable or bigquery

Why ?

  • first bigtable is not a database made for analytics. Or maybe you will have a way to format your data so that it can be partitioned and fitted into small rows.

  • bigquery has a cost and I would say it is not the right tool for the job, despite being able to handle the load. It will work but given the datasize and the price I would say you don’t need it.

I would better launch a clickhouse install in order to prevent vendor lockdown https://clickhouse-dashboard.cube.dev/how-to-run-click-house

READ MORE
3 upvotes·2 comments·3.3K views
zack5676
zack5676
·
August 23rd 2023 at 6:45AM

Clickhouse runs analytical queries super fast from my experience, assuming data is structured well for the columnar storage type cases it handles well (which are many analytics queries, but less likely pulling out stuff from JSON, etc).

·
Reply
Facts Rule
Facts Rule
·
August 26th 2021 at 2:16AM

Thank you for your feedback!

·
Reply