What is Diamond and what are its top alternatives?
Diamond alternatives & related posts
related Graphite posts
Why we spent several years building an open source, large-scale metrics alerting system, M3, built for Prometheus:
By late 2014, all services, infrastructure, and servers at Uber emitted metrics to a Graphite stack that stored them using the Whisper file format in a sharded Carbon cluster. We used Grafana for dashboarding and Nagios for alerting, issuing Graphite threshold checks via source-controlled scripts. While this worked for a while, expanding the Carbon cluster required a manual resharding process and, due to lack of replication, any single node’s disk failure caused permanent loss of its associated metrics. In short, this solution was not able to meet our needs as the company continued to grow.
To ensure the scalability of Uber’s metrics backend, we decided to build out a system that provided fault tolerant metrics ingestion, storage, and querying as a managed platform...
(GitHub : https://github.com/m3db/m3)
A huge part of our continuous deployment practices is to have granular alerting and monitoring across the platform. To do this, we run Sentry on-premise, inside our VPCs, for our event alerting, and we run an awesome observability and monitoring system consisting of StatsD, Graphite and Grafana. We have dashboards using this system to monitor our core subsystems so that we can know the health of any given subsystem at any moment. This system ties into our PagerDuty rotation, as well as alerts from some of our Amazon CloudWatch alarms (we’re looking to migrate all of these to our internal monitoring system soon).
related Node.js posts
Winds 2.0 is an open source Podcast/RSS reader developed by Stream with a core goal to enable a wide range of developers to contribute.
When I joined NYT there was already broad dissatisfaction with the LAMP (Linux Apache HTTP Server MySQL PHP) Stack and the front end framework, in particular. So, I wasn't passing judgment on it. I mean, LAMP's fine, you can do good work in LAMP. It's a little dated at this point, but it's not ... I didn't want to rip it out for its own sake, but everyone else was like, "We don't like this, it's really inflexible." And I remember from being outside the company when that was called MIT FIVE when it had launched. And been observing it from the outside, and I was like, you guys took so long to do that and you did it so carefully, and yet you're not happy with your decisions. Why is that? That was more the impetus. If we're going to do this again, how are we going to do it in a way that we're gonna get a better result?
So we're moving quickly away from LAMP, I would say. So, right now, the new front end is React based and using Apollo. And we've been in a long, protracted, gradual rollout of the core experiences.
React is now talking to GraphQL as a primary API. There's a Node.js back end, to the front end, which is mainly for server-side rendering, as well.
Behind there, the main repository for the GraphQL server is a big table repository, that we call Bodega because it's a convenience store. And that reads off of a Kafka pipeline.
related ASP.NET posts
Finding the most effective dev stack for a solo developer. Over the past year, I've been looking at many tech stacks that would be 'best' for me, as a solo, indie, developer to deliver a desktop app (Windows & Mac) plus mobile - iOS mainly. Initially, Xamarin started to stand-out. Using .NET Core as the run-time, Xamarin as the native API provider and Xamarin Forms for the UI seemed to solve all issues. But, the cracks soon started to appear. Xamarin Forms is mobile only; the Windows incarnation is different. There is no Mac UI solution (you have to code it natively in Mac OS Storyboard. I was also worried how Xamarin Forms , if I was to use it, was going to cope, in future, with Apple's new SwiftUI and Google's new Fuchsia.
This plethora of techs for the UI-layer made me reach for the safer waters of using Web-techs for the UI. Lovely! Consistency everywhere (well, mostly). But that consistency evaporates when platform issues are addressed. There are so many web frameworks!
But, I made a simple decision. It's just me...I am clever, but there is no army of coders here. And I have big plans for a business app. How could just 1 developer go-on to deploy a decent app to Windows, iPhone, iPad & Mac OS? I remembered earlier days when I've used Microsoft's ASP.NET to scaffold - generate - loads of Code for a web-app that I needed for several charities that I worked with. What 'generators' exist that do a lot of the platform-specific rubbish, allow the necessary customisation of such platform integration and provide a decent UI?
I've placed my colours to the Quasar Framework mast. Oh dear, that means Electron desktop apps doesn't it? Well, Ive had enough of loads of Developers saying that "the menus won't look native" or "it uses too much RAM" and so on. I've been using non-native UI-wrapped apps for ages - the date picker in Outlook on iOS is way better than the native date-picker and I'd been using it for years without getting hot under the collar about it. Developers do get so hung-up on things that busy Users hardly notice; don't you think?. As to the RAM usage issue; that's a bit true. But Users only really notice when an app uses so much RAM that the machine starts to page-out. Electron contributes towards that horizon but does not cause it. My Users will be business-users after all. Somewhat decent machines.
Looking forward to all that lovely Vue.js around my TypeScript and all those really, really, b e a u t I f u l UI controls of Quasar Framework . Still not sure that 1 dev can deliver all that... but I'm up for trying...
I found Heroku to be a great option to get ExpressJS up and running with very little hustle. The free tier is great, but I'd recommend to set up a cronjob to visit your site every few minutes so that the server stays awake. Netlify was the option to host the front-end because doing the server side rendering on #Heroku would have taken a little more time than I'd like to. For the moment pre-rendering the app with prerender-spa-plugin is enough to help with #seo. Puppeteer was my choice over other options because it made it easier to scrape websites made on ASP.NET which is what I needed in this case. And Vue.js is my top choice at the moment because it's really beginner friendly and it has a lot of the features I like about Angular 2 and React. vuex is a must in most of the app I build.
related Rails posts
But wowza, things have changed. Tooling is just way, way better. I'm primarily web-oriented, and using React and Apollo together the past few years really opened my eyes to building rich apps. And I deeply apologize for using the phrase rich apps; I don't think I've ever said such Enterprisey words before.
But yeah, things are different now. I still love Rails, and still use it for a lot of apps I build. But it's that silly rich apps phrase that's the problem. Users have way more comprehensive expectations than they did even five years ago, and the JS community does a good job at building tools and tech that tackle the problems of making heavy, complicated UI and frontend work.
Back in 2014, I was given an opportunity to re-architect SmartZip Analytics platform, and flagship product: SmartTargeting. This is a SaaS software helping real estate professionals keeping up with their prospects and leads in a given neighborhood/territory, finding out (thanks to predictive analytics) who's the most likely to list/sell their home, and running cross-channel marketing automation against them: direct mail, online ads, email... The company also does provide Data APIs to Enterprise customers.
I had inherited years and years of technical debt and I knew things had to change radically. The first enabler to this was to make use of the cloud and go with AWS, so we would stop re-inventing the wheel, and build around managed/scalable services.
For the SaaS product, we kept on working with Rails as this was what my team had the most knowledge in. We've however broken up the monolith and decoupled the front-end application from the backend thanks to the use of Rails API so we'd get independently scalable micro-services from now on.
Our various applications could now be deployed using AWS Elastic Beanstalk so we wouldn't waste any more efforts writing time-consuming Capistrano deployment scripts for instance. Combined with Docker so our application would run within its own container, independently from the underlying host configuration.
Storage-wise, we went with Amazon S3 and ditched any pre-existing local or network storage people used to deal with in our legacy systems. On the database side: Amazon RDS / MySQL initially. Ultimately migrated to Amazon RDS for Aurora / MySQL when it got released. Once again, here you need a managed service your cloud provider handles for you.
Future improvements / technology decisions included:
Caching: Amazon ElastiCache / Memcached CDN: Amazon CloudFront Systems Integration: Segment / Zapier Data-warehousing: Amazon Redshift BI: Amazon Quicksight / Superset Search: Elasticsearch / Amazon Elasticsearch Service / Algolia Monitoring: New Relic
As our usage grows, patterns changed, and/or our business needs evolved, my role as Engineering Manager then Director of Engineering was also to ensure my team kept on learning and innovating, while delivering on business value.
One of these innovations was to get ourselves into Serverless : Adopting AWS Lambda was a big step forward. At the time, only available for Node.js (Not Ruby ) but a great way to handle cost efficiency, unpredictable traffic, sudden bursts of traffic... Ultimately you want the whole chain of services involved in a call to be serverless, and that's when we've started leveraging Amazon DynamoDB on these projects so they'd be fully scalable.
related Django posts
Simple controls over complex technologies, as we put it, wouldn't be possible without neat UIs for our user areas including start page, dashboard, settings, and docs.
Initially, there was Django. Back in 2011, considering our Python-centric approach, that was the best choice. Later, we realized we needed to iterate on our website more quickly. And this led us to detaching Django from our front end. That was when we decided to build an SPA.
For building user interfaces, we're currently using React as it provided the fastest rendering back when we were building our toolkit. It’s worth mentioning Uploadcare is not a front-end-focused SPA: we aren’t running at high levels of complexity. If it were, we’d go with Ember.js.
However, there's a chance we will shift to the faster Preact, with its motto of using as little code as possible, and because it makes more use of browser APIs. One of our future tasks for our front end is to configure our Webpack bundler to split up the code for different site sections. For styles, we use PostCSS along with its plugins such as cssnano which minifies all the code.
All that allows us to provide a great user experience and quickly implement changes where they are needed with as little code as possible.
Possible pros for Python / Django: - easy syntax, easier to learn for me as a beginner - fast development, earlier release - libraries for mathematical and scientific computation
Which software would you use in my case? Are my arguments for Python/NodeJS right? Which kind of database would you use?
Thank you for your answer!
related Laravel posts
I use Laravel because it's the most advances PHP framework out there, easy to maintain, easy to upgrade and most of all : easy to get a handle on, and to follow every new technology ! PhpStorm is our main software to code, as of simplicity and full range of tools for a modern application.
Google Analytics Analytics of course for a tailored analytics, Bulma as an innovative CSS framework, coupled with our Sass (Scss) pre-processor.
To deploy, we set up Buddy to easily send the updates on our nginx / Ubuntu server, where it will connect to our GitHub Git private repository, pull and do all the operations needed with Deployer .
CloudFlare ensure the rapidity of distribution of our content, and Let's Encrypt the https certificate that is more than necessary when we'll want to sell some products with our Stripe api calls.
Asana is here to let us list all the functionalities, possibilities and ideas we want to implement.
Back at the start of 2017, we decided to create a web-based tool for the SEO OnPage analysis of our clients' websites. We had over 2.000 websites to analyze, so we had to perform thousands of requests to get every single page from those websites, process the information and save the big amounts of data somewhere.
Very soon we realized that the initial chosen script language and database, PHP, Laravel and MySQL, was not going to be able to cope efficiently with such a task.
By that time, we were doing some experiments for other projects with a language we had recently get to know, Go , so we decided to get a try and code the crawler using it. It was fantastic, we could process much more data with way less CPU power and in less time. By using the concurrency abilites that the language has to offers, we could also do more Http requests in less time.
Unfortunately, I have no comparison numbers to show about the performance differences between Go and PHP since the difference was so clear from the beginning and that we didn't feel the need to do further comparison tests nor document it. We just switched fully to Go.
There was still a problem: despite the big amount of Data we were generating, MySQL was performing very well, but as we were adding more and more features to the software and with those features more and more different type of data to save, it was a nightmare for the database architects to structure everything correctly on the database, so it was clear what we had to do next: switch to a NoSQL database. So we switched to MongoDB, and it was also fantastic: we were expending almost zero time in thinking how to structure the Database and the performance also seemed to be better, but again, I have no comparison numbers to show due to the lack of time.
As of now, we don't only use the tool intern but we also opened it for everyone to use for free: https://tool-seo.com
related Android SDK posts
We are using React Native in #SmartHome to share the business logic between Android and iOS team and approach users with a unique brand experience. The drawback is that we require lots of native Android SDK and Objective-C modules, so a good part of the invested time is there. The gain for a app that relies less on native communication, sensors and OS tools should be even higher.
We use a microservices structure on top of Zeit's @now that read from firebase. We use JWT auth to authenticate requests among services and from users, following GitHub philosophy of using the same infrastructure than its API consumers. Firebase is used mainly as a key-value store between services and as a backup database for users. We also use its authentication mechanisms.
You can be super locked-in if you also rely on it's analytics, but we use Amplitude for that, which offers us great insights. Intercom for communications with end-user and Mailjet for marketing.
I've recently switched to using Expo for initializing and developing my React Native apps. Compared to React Native CLI, it's so much easier to get set up and going. Setting up and maintaining Android Studio, Android SDK, and virtual devices used to be such a headache. Thanks to Expo, I can now test my apps directly on my Android phone, just by installing the Expo app. I still use Xcode Simulator for iOS testing, since I don't have an iPhone, but that's easy anyway. The big win for me with Expo is ease of Android testing.
The Expo SDK also provides convenient features like Facebook login,
MapView, push notifications, and many others. https://docs.expo.io/versions/v31.0.0/sdk/
related Spring Boot posts
We are in the process of building a modern content platform to deliver our content through various channels. We decided to go with Microservices architecture as we wanted scale. Microservice architecture style is an approach to developing an application as a suite of small independently deployable services built around specific business capabilities. You can gain modularity, extensive parallelism and cost-effective scaling by deploying services across many distributed servers. Microservices modularity facilitates independent updates/deployments, and helps to avoid single point of failure, which can help prevent large-scale outages. We also decided to use Event Driven Architecture pattern which is a popular distributed asynchronous architecture pattern used to produce highly scalable applications. The event-driven architecture is made up of highly decoupled, single-purpose event processing components that asynchronously receive and process events.
To build our #Backend capabilities we decided to use the following: 1. #Microservices - Java with Spring Boot , Node.js with ExpressJS and Python with Flask 2. #Eventsourcingframework - Amazon Kinesis , Amazon Kinesis Firehose , Amazon SNS , Amazon SQS, AWS Lambda 3. #Data - Amazon RDS , Amazon DynamoDB , Amazon S3 , MongoDB Atlas
To build #Webapps we decided to use Angular 2 with RxJS
#Devops - GitHub , Travis CI , Terraform , Docker , Serverless
Google Analytics Elasticsearch Amazon Route 53DevOps
GitHub Docker Webpack CircleCI Jenkins Travis CI Gradle Apache MavenCooperation Tools
Jira notion.so Trello
related .NET posts
The core Web application of Raygun is still a Microsoft ASP.NET MVC application. Not too much has changed from a fundamental technology standpoint. We originally built using Mono, which just bled memory and would need to be constantly recycled. So we looked around at the options and what would be well suited to the highly transactional nature of our API. We settled on Node.js, feeling that the event loop model worked well given the lightweight workload of each message being processed. This served us well for several years.
When we started to look at .NET Core in early 2016, it became quite obvious that being able to asynchronously hand off to our queuing service greatly improved throughput. Unfortunately, at the time, Node.js didn’t provide an easy mechanism to do this, while .NET Core had great concurrency capabilities from day one. This meant that our servers spent less time blocking on the hand off, and could start processing the next inbound message. This was the core component of the performance improvement.
We chose .NET because it was a platform that our team was familiar with. Also we were skilled enough with it to know many performance tips and tricks to get the most from it. Due to this experience, it helped us get to market faster and deliver great performance.
When starting a new company and building a new product w/ limited engineering we chose to optimize for expertise and rapid development, landing on Rails API, w/ AngularJS on the front.
The reality is that we're building a CRUD app, so we considered going w/ vanilla Rails MVC to optimize velocity early on (it may not be sexy, but it gets the job done). Instead, we opted to split the codebase to allow for a richer front-end experience, focus on skill specificity when hiring, and give us the flexibility to be consumed by multiple clients in the future.
We also considered .NET core or Node.js for the API layer, and React on the front-end, but our experiences dealing with mature Node APIs and the rapid-fire changes that comes with state management in React-land put us off, given our level of experience with those tools.
We're using GitHub and Trello to track issues and projects, and a plethora of other tools to help the operational team, like Zapier, MailChimp, Google Drive with some basic Vue.js & HTML5 apps for smaller internal-facing web projects.
related Symfony posts
I really love Django because it is really fast to create a web application from scratch and it has a lot a facilities like the ORM or the Admin module ! The Python language is really easy to read and powerful, that's why I prefer Django over Symfony.
I use Django at work to make tools for the technicians but I also use it for me to build my personal website which I host on PythonAnywhere.
We needed our e-commerce platform (built using WooCommerce) to be able to keep products in sync with our #pim (provided by #akeneo) which is built in Symfony . We hooked into the kernel.event_listener to send RabbitMQ messages to a WordPress API endpoint that triggers the updated product to rebuild with fresh data.
related Meteor posts
Mixmax was originally built using Meteor as a single monolithic app. As more users began to onboard, we started noticing scaling issues, and so we broke out our first microservice: our Compose service, for writing emails and Sequences, was born as a Node.js service. Soon after that, we broke out all recipient searching and storage functionality to another Node.js microservice, our Contacts service. This practice of breaking out microservices in order to help our system more appropriately scale, by being more explicit about each microservice’s responsibilities, continued as we broke out numerous more microservices.
As Mixmax began to scale super quickly, with more and more customers joining the platform, we started to see that the Meteor app was still having a lot of trouble scaling due to how it tried to provide its reactivity layer. To be honest, this led to a brutal summer of playing Galaxy container whack-a-mole as containers would saturate their CPU and become unresponsive. I’ll never forget hacking away at building a new microservice to relieve the load on the system so that we’d stop getting paged every 30-40 minutes. Luckily, we’ve never had to do that again! After stabilizing the system, we had to build out two more microservices to provide the necessary reactivity and authentication layers as we rebuilt our Meteor app from the ground up in Node.js. This also had the added benefit of being able to deploy the entire application in the same AWS VPCs. Thankfully, AWS had also released their ALB product so that we didn’t have to build and maintain our own websocket layer in Amazon EC2. All of our microservices, except for one special Go one, are now in Node with an nginx frontend on each instance, all behind AWS Elastic Load Balancing (ELB) or ALBs running in AWS Elastic Beanstalk.
related Django REST framework posts
Zulip has been powered by Django since the very early days of its development with Django 1.4, back in 2012. As a reasonably mature web application with significant scale, we're at the stage in many companies' development where one starts to rip out more and more of the web framework to optimize things or just make them work the way we want. (E.g. while I was at Dropbox in early 2016, we discovered we only had about 600 lines of code left from the original Pylons framework that actually ran).
One of the things that has been really fantastic about Django is that we're still happily using it for the vast majority of code in the project, and every time Django comes out with a new release, I read the changelog and get excited about several improvements that actually make my life better. While Django has made some design decisions that I don't agree with (e.g. I'm not a fan of Django REST framework, and think it makes life more difficult), Django also makes it easy to do your own thing, which we've done to great effect (see the linked article for details on our
Overall I think we've gotten a ton of value out of Python and Django and would recommend it to anyone starting a new full-featured web application project today.
related Next.js posts
We are in the process of adopting Next.js as our React framework and using Storybook to help build our React components in isolation. This new part of our frontend is written in TypeScript, and we use Emotion for CSS/styling. For delivering data, we use GraphQL and Apollo. Jest, Percy, and Cypress are used for testing.
I just finished a web app meant for a business that offers training programs for certain professional courses. I chose this stack to test out my skills in graphql and react. I used Node.js , GraphQL , MySQL for the #Backend utilizing Prisma as a database interface for MySQL to provide CRUD APIs and graphql-yoga as a server. For the #frontend I chose React, styled-components for styling, Next.js for routing and SSR and Apollo for data management. I really liked the outcome and I will definitely use this stack in future projects.
related Play posts
Some may wonder why did we choose Grails ? Really good question :) We spent quite some time to evaluate what framework to go with and the battle was between Play Scala and Grails ( Groovy ). We have enough experience with both and, to be honest, I absolutely in love with Scala; however, the tipping point for us was the potential speed of development. Grails allows much faster development pace than Play , and as of right now this is the most important parameter. We might convert later though. Also, worth mentioning, by default Grails comes with Gradle as a build tool, so why change?
related Yii posts
Of all PHP frameworks, my best and only choice is Yii . Think of this: you have a MySQL database, it contains several tables. Now you want to setup a PHP-MVC site, firstly, you must create Models, Yii have a very handy tool called Gii, you can easily create model with Gii just by one click, Gii will read your database table columns and create PHP models automatically for you. Now you need Controller, still with Gii, it will automatically create all 4 php files for you with Insert/Delete/Update/Select even with Search function.
Well, now the most modern way is to have a RESTful API, that's even easier with Yii, you even don't need to care about all the columns, just 4 lines of code you can expose your database table as RESTful API with all GET/POST/PUT/DELETE support, even you change your database table columns, you don't need to change any PHP code.
For security, Yii have embedded authentication and RBAC support. For multi language, Yii have embedded i18n support, all with out-of-box. Just play with it, I bet you will love it.