PySpark vs Python

Need advice about which tool to choose?Ask the StackShare community!

PySpark

256
286
+ 1
0
Python

237.2K
193.6K
+ 1
6.8K
Add tool

PySpark vs Python: What are the differences?

Introduction

In this article, we will discuss the key differences between PySpark and Python. Both PySpark and Python are popular programming languages used in the field of data analysis and processing. While both languages have their own merits, they also differ in several aspects. Let's explore the differences between PySpark and Python in more detail.

  1. Data parallelism: One of the major differences between PySpark and Python is the approach to handle data parallelism. PySpark is specifically designed to handle big data processing and provides built-in support for distributed computing using Apache Spark. It allows the execution of tasks in parallel across multiple machines, making it more efficient for large-scale data processing. On the other hand, Python is primarily a general-purpose programming language and does not have native support for distributed processing. It can still handle parallelism using libraries like multiprocessing, but it may not be as efficient as PySpark for big data processing.

  2. Performance: Another significant difference between PySpark and Python is their performance. PySpark, being built on top of Apache Spark, is optimized for handling large datasets and can perform complex operations like data joins and aggregations more efficiently. It achieves this by utilizing in-memory computing and intelligent data partitioning. Python, on the other hand, may not perform as efficiently as PySpark for big data processing tasks. While Python is a versatile language suitable for many purposes, its performance may degrade when dealing with large datasets and complex computations.

  3. Scalability: PySpark provides excellent scalability capabilities due to its distributed computing nature. It can easily scale its processing to handle massive amounts of data without any significant performance degradation. Python, on the other hand, may face scalability limitations when dealing with huge datasets or when running on a single machine. Though Python can leverage libraries like NumPy and Pandas for efficient numerical and data operations, it may still struggle to scale seamlessly like PySpark.

  4. Data processing techniques: PySpark and Python also differ in terms of the data processing techniques they support. PySpark offers a wide range of tools and APIs for distributed data processing, including batch processing, real-time streaming, and machine learning capabilities. It provides a unified API to interact with data stored in various formats such as SQL tables, Hadoop Distributed File System (HDFS), and other data storage systems supported by Apache Spark. Python, on the other hand, primarily relies on libraries like Pandas, NumPy, and scikit-learn for data manipulation and analysis. While Python can handle various data processing tasks efficiently, it does not provide the same level of distributed computing capabilities as PySpark.

  5. Community and ecosystem: PySpark and Python also differ in terms of their community and ecosystem support. Python has a massive community and an extensive ecosystem of libraries and packages that cover a wide range of domains. This makes Python a versatile language with a rich set of tools for data analysis, machine learning, and other areas. On the other hand, PySpark has a smaller community compared to Python but is supported by the Apache Software Foundation, ensuring continuous development and improvement. It has a growing ecosystem of libraries and packages specifically tailored for distributed data processing.

  6. Learning curve and ease of use: Lastly, PySpark and Python differ in terms of their learning curve and ease of use. Python has a relatively gentle learning curve and is widely regarded as an easy-to-learn language, making it accessible to beginners. It has a straightforward syntax and offers a lot of resources, tutorials, and documentation for beginners. PySpark, on the other hand, has a steeper learning curve and requires a deeper understanding of distributed computing concepts. It also requires familiarity with Apache Spark's APIs and concepts, which can be challenging for beginners without prior experience in big data processing.

In summary, PySpark and Python differ in their approach to data parallelism, performance, scalability, supported data processing techniques, community and ecosystem support, as well as the learning curve and ease of use. While PySpark is more suitable for big data processing and distributed computing, Python is a versatile language suitable for a wide range of tasks, including data analysis and machine learning.

Advice on PySpark and Python
Needs advice
on
Node.jsNode.jsPHPPHP
and
PythonPython

Hi, I have a project on my mind, and I need some help. First of all, I know it is all about personal preference, but I am a beginner in the back-end part. So, I am trying to figure out which language is better, for example, for user authentication and interaction between the users. Also, I don't know which framework is better for this work. My first thought was to use PHP, but after some research on the internet, I'm leaning towards Laravel. I will be grateful if you have some advice for me.

#newbie

See more
Replies (7)
Recommends
on
Node.jsNode.jsPHPPHP

First thoughts: * As a beginner you need to understand concepts first, all languages out there are great, each has it's own philosophy, each is better suited for a specific situation. Learn concepts first, do something, then you will understand the differences between them and why one should be chose over another for a task * As a project manager you want your project to come to an end. You will get lost in all the different solutions out there (and it's good), but don't get lost too far. Very often I see people getting lost in those debates and never achieving things, like someone writing a book that would still be choosing the font 6 months after (we've all done that it's ok, but we have to realise it)

Chances are your project can be equally good on any different stacks. I heard an interview of someone from Uber who said something like they started with python, went to node, went back to python and went to go, and with micro-services now they can have all of them all-together.

Last remark: from what I know Laravel is a framework for PHP, so it IS PHP. Just like Symphony for PHP, Express for Node.js, Koa for Node.js, Flask for Python ...

Now to answer your question :

  • PHP has a big community, it is great and easy to start with, and you will definitely will learn real object oriented structure
  • Node.js has a big community too, don't worry finding help will be as easy. It is less easy to start with but in my point of view it is a lot easier to keep on going with it on a long run. Why ? Because it's very easy to run a new project, and it executes javascript. How is it good ? Because chances are that your front will also be using javascript (React.js / View.js are crazy good). Thanks to that you will be able to master the language better because you will use it all day (and at first mastering one language is more valuable than barely knowing two) and you won't have to switch languages in your head when you code. And communication between front and back will be in json ... Which is crazy close to javascript.

Alexander is right, if you go with PHP take your time first to do things by yourself like building your own MVC, the benefit is huge and the risk is to never really be able to understand what's happening on a deeper level. (at some point you can switch to a framework though). He's also right on choosing a strongly typed language, problem is javascript is not. This is why, if you choose node, when you start being confident, add typesccript.

Hope it helps, good luck

See more
Octavian Irimia
Recommends
on
PHPPHP

Short answer, if it's a web project (and I guess it is) go with PHP and you can integrate NodeJs services later.

@adzaria (Ezra Fayet) gave a great answer and I'd like to emphasize the first part: As a beginner you need to understand concepts first. For me that means to understand the web, how servers and requests work, APIs and few others.

Now, I'd like to add few things so, this is the long answer:

Why PHP?

  • Everyone knows about the community - PHP is way older so you will find lots of resources and I am not only talking about learning - also lots of helpful tools and packages
  • PHP is great for OOP - not perfect, but with PHP7 got great - and if you are a beginner you want to know good OOP for your future. Let's say JavaScript's OOP is a bit strange; I will not get into details but, let's say "it's not by the book". You can still learn JavaScript for your front-end

Why not Python? Python got popular because of AI - don't use PHP for AI and don't use Python for web applications. I can elaborate a lot here but I guess you get the point.

Why not NodeJs?

  • NodeJs got popular because of sockets - and it works great, but as a service
  • Try to find a good and affordable hosting for NodeJs. How about for Python?
  • I would not ignore the security issues that it had and could appear. PHP is older and, therefore, wiser :)

Now, about a framework... is this a learning project or something that you need to do fast? My advice is to start a small project and not use any framework. However, you can use packages and inspire from a framework's architecture - Laravel is a good role model.

Why not start a big project? You will get distracted, get into details and product design stuff and get scared or border and abandon it. For your project you need an MVP - list of minimum required features that you put on paper - that you will complete. After that you can improve.

Good luck!

See more
somes kumar k
Member Technical Staff at Manage Engine, division of Zoho Corp · | 3 upvotes · 331.6K views
Recommends
on
Node.jsNode.js

you can choose Node.js Here are my points

Node.js is build over chrome’s v8 and its works on non blocking io. Node.js have huge community and great packages (npm) to help you out in most cases and makes development faster Node.js has been adopted by many multi dollar company Hope this helps😊

See more
Tarun Batra
Senior Software Developer at Okta · | 2 upvotes · 320K views
Recommends
on
DjangoDjangoPythonPython

Python, PHP and Node.js all are capable of being used to create good complex software. There are many examples of similar applications built on all of them. If I have to pick one, I would say consider Python and Django. It is fairly easy to develop web applications on top of this stack. Scaling and maintaining the application should also not be a problem given a lot of resources are available online.

See more
Recommends
on
DjangoDjangoPythonPython

I have used Laravel, but with Django you can develop faster, as authentication and admin panel are configured out of the box. It users SQLite by default and you won't have to worry about the database in the begginning

See more
Alexander Santos
Fullstack Developer at 3CON · | 2 upvotes · 327.4K views
Recommends
on
PHPPHP

The reason why i chose PHP is the amount of content you can find on the internet easily. As you quoted being a beginner, i think a more mature language would be better. And that's also another reason for following with PHP.

Python is simple and "mature", but it can be a bit hard to understand if you are a beginner. Python relies on heavy abstraction, and that's the reason behind it's simplicity. Python is an "easy to play, hard to master" language, i never recommend it to beginners. Also, one [maybe personal] reason why i don't like to use Python as back-end is: Python is very data-focused. So if your app has focus on business logic, Python wouldn't fit very well. And with that becomes an advantage, if your app has statistical focus, being data-focused or something like that, Python has huge advantage among all other languages due to many great tools the community has built.

About Node, it's like PHP, but less mature. It's as easy as PHP to find tools that can help you, for example, to abstract the database-connection's logic. But to find architectural-focused content, more advanced concepts, it's a lot harder. While that, Laravel's community, for example, has a lot of materials that involves those concepts.

Still, if you are really a beginner, i don't recommend using Laravel with PHP. Do things on plain PHP first, understand the reason behind using frameworks and Laravel's motivation.

Also, consider a strong-typed language first, those are considered more didatic, but less flexible.

See more
Recommends
on
Node.jsNode.jsPHPPHPPythonPython

You cannot choose between Python | PHP | NodeJS Since they are entirely for Different purpose.

In Bird view

Python - Large Scale Projects and if you want a job in big IT company.

Node.JS - Huge computing projects and if you want job in Silicon valley startup.

PHP - Cost Effective and If you want start a business in near future.

See more
Kamal Makroum
Needs advice
on
JavaJavaPythonPython
and
ReactReact

Hi everyone.

I am willing to build a used car sales platform, which will have a lot of stock/photos and will rely a lot on the back end functions and data generating. Java seems to be a good choice, but what other options can I consider that can also be easily scalable as well as a little faster to write?

Thank you

See more
Replies (2)
Recommends

Firstly, you must know that java and python are both amazing languages. But I recommend python mainly because of the variety of modules and packages available to do almost anything. If you are planning on adding graphs, you can use the matplotlib library and to add photos, use the pillow module. And just note that both of these aren't available by default, so you need to install them through pip.

See more
Ruslan Rayanov
Recommends

Hi, Kamal! I don't know if your question is still relevant. But I would like to introduce you to our solution, perhaps it will be useful for future projects. We have developed a web application constructor that can be used to create almost any website or application https://falconspace.site/. The entire development stack is reduced to SQL only. The platform is easy to configure and make subsequent changes if necessary.

See more
Needs advice
on
JavaScriptJavaScriptPythonPython
and
RustRust

So, I've been working with all 3 languages JavaScript, Python and Rust, I know that all of these languages are important in their own domain but, I haven't took any of it to the point where i could say I'm a pro at any of these languages. I learned JS and Python out of my own excitement, I learned rust for some IoT based projects. just confused which one i should invest my time in first... that does have Job and freelance potential in market as well...

I am an undergraduate in computer science. (3rd Year)

See more
Replies (3)
Recommends
on
JavaScriptJavaScript

I would start focusing on Javascript because even working with Rust and Python, you're always going to encounter some Javascript for front-ends at least. It has: - more freelancing opportunities (starting to work short after a virus/crisis, that's gonna help) - can also do back-end if needed (I would personally avoid specializing in this since there's better languages for the back-end part) - hard to avoid. it's everywhere and not going away (well not yet)

Then, later, for back-end programming languages, Rust seems like your best bet. Its pros: - it's satisfying to work with (after the learning curve) - it's got potential to grow big in the next year (also with better paying jobs) - it's super versatile (you can do high-perf system stuff, graphics, ffi, as well as your classic api server) It comes with a few cons though: - it's harder to learn (expect to put in years) - the freelancing options are virtually non-existent (and I would expect them to stay limited, as rust is better for long-term software than prototypes)

See more
Recommends
on
JavaScriptJavaScript

I suggest you to go with JavaScript. From my perspective JavaScript is the language you should invest your time in. The community of javascript and lots of framework helps developer to build what they want to build in no time whether it a desktop, web, mobile based application or even you can use javascript as a backend as well. There are lot of frameworks you can start learning i suggest you to go with (react,vue) library both are easy to learn than angular which is a complete framework.

And if you want to go with python as a secondary tool then i suggest you to learn a python framework (Flask,Django).

See more
Moinul Moin
Recommends
on
JavaScriptJavaScript

go for javascript, brother.

See more
Needs advice
on
JSONJSON
and
PythonPython

Hi. Currently, I have a requirement where I have to create a new JSON file based on the input CSV file, validate the generated JSON file, and upload the JSON file into the application (which runs in AWS) using API. Kindly suggest the best language that can meet the above requirement. I feel Python will be better, but I am not sure with the justification of why python. Can you provide your views on this?

See more
Replies (3)
Recommends
on
PythonPython

Python is very flexible and definitely up the job (although, in reality, any language will be able to cope with this task!). Python has some good libraries built in, and also some third party libraries that will help here. 1. Convert CSV -> JSON 2. Validate against a schema 3. Deploy to AWS

  1. The builtins include json and csv libraries, and, depending on the complexity of the csv file, it is fairly simple to convert:
import csv
import json

with open("your_input.csv", "r") as f:
    csv_as_dict = list(csv.DictReader(f))[0]

with open("your_output.json", "w") as f:
    json.dump(csv_as_dict, f)
  1. The validation part is handled nicely by this library: https://pypi.org/project/jsonschema/ It allows you to create a schema and check whether what you have created works for what you want to do. It is based on the json schema standard, allowing annotation and validation of any json

  2. It as an AWS library to automate the upload - or in fact do pretty much anything with AWS - from within your codebase: https://aws.amazon.com/sdk-for-python/ This will handle authentication to AWS and uploading / deploying the file to wherever it needs to go.

A lot depends on the last two pieces, but the converting itself is really pretty neat.

See more
Recommends
on
GolangGolang

I would use Go. Since CSV files are flat (no hierarchy), you could use the encoding/csv package to read each row, and write out the values as JSON. See https://medium.com/@ankurraina/reading-a-simple-csv-in-go-36d7a269cecd. You just have to figure out in advance what the key is for each row.

See more
Max Musing
Founder & CEO at BaseDash · | 1 upvotes · 286.8K views
Recommends
on
Node.jsNode.js
at

This should be pretty doable in any language. Go with whatever you're most familiar with.

That being said, there's a case to be made for using Node.js since it's trivial to convert an object to JSON and vice versa.

See more
Needs advice
on
JavaJavaLuaLua
and
PythonPython

I am trying to make Roblox game which requires Lua. I quite don't want to go with Lua just because other tools just might let me do more projects later on. I heard that Python is most similar to Lua, but I am still not sure which tool to use. Java, I think it will help me with many stuff later on for websites, projects, and more!

See more
Replies (2)
Rafey Iqbal Rahman
Recommends
on
LuaLua
at

Since you are trying to make a Roblox game, you have no other option than to use Lua, since Roblox only allows coding in Lua. Yes, you've heard right, Python is identical and as easy as Lua, although Lua is easier than Python. Beginning from Lua and then escalating to Python is recommended. Java is only helpful when you are creating a heavy, big-budget, enterprise-level product, otherwise, Python would suffice.

See more
Recommends
on
TypeScriptTypeScript

If you really hate lua check out roblox-ts, a tool that compiles typescript code into roblox lua. https://github.com/roblox-ts/roblox-ts

See more
Needs advice
on
JavaJavaPHPPHP
and
PythonPython

Hi everyone, I have just started to study web development, so I'm very new in this field. I would like to ask you which tools are most updated and good to use for getting a job in medium-big company. Front-end is basically not changing by time so much (as I understood by researching some info), so my question is about back-end tools. Which backend tools are most updated and requested by medium-big companies (I am searching for immediate job possibly)?

Thank you in advance Davit

See more
Replies (4)
Pierrick Martos
Engineering Manager at Akeneo · | 20 upvotes · 319.4K views
Recommends
on
PythonPython

Go with Python definetly. It's used everywhere by web developers for backend developments : API, website backend, workers... but also by data scientists (lot lot of resources, models and libraries in Python it's language #1). For the web parts, best web framework are in Python : https://stackshare.io/microframeworks (Flask #2 and Django #3). Java is good but trend is not great in terms of popularity amongs developers and tech leaders.

See more
Vijayakumar Rajagopal
Recommends
on
JavaJava

As per my experience java is most wanted for web development as of now. micro service is evolving . with frameworks like spring boot supports rapid development. Spring boot + Docker + kubernetes great combination.

See more
sharik zama
Software engineering Intern at EPAM Systems · | 5 upvotes · 318.5K views
Recommends
on
JavaScriptJavaScript

I would recommend learning HTML, CSS, and JavaScript (most important). JavaScript forms the backbone of web development. And, there are many popular and widely used frameworks like Angular and React that heavily rely on the knowledge of JavaScript. The number of job opportunities are much more when it comes to javascript.

See more
Chathuranga Bandara
Recommends
on
PythonPython

I would recommend Python as the programming language and as you are a new developer, Flask to start with. It gives you a solid understanding on the web patterns such as REST and will get you up and running in no time. However, I suggest you to read and study on front-end technologies like (React or Vue) and databases (SQL and NoSQL) and probably some NodeJS as well. First grasp the concepts (which Python is ideal for) then it does not really matter the language as such.

See more
Needs advice
on
GolangGolangPythonPython
and
React NativeReact Native

I've been juggling with an app idea and am clueless about how to build it.

A little about the app:

  • Social network type app ,
  • Users can create different directories, in those directories post images and/or text that'll be shared on a public dashboard .

Directory creation is the main point of this app. Besides there'll be rooms(groups),chatting system, search operations similar to instagram,push notifications

I have two options:

  1. React Native, Python, AWS stack or
  2. Flutter, Go ( I don't know what stack or tools to use)
See more
Replies (6)
George Krachtopoulos
Recommends
on
PythonPython

Currently, I have decided to use Python and JavaScript (especially React and Node.js) for any of my projects. Well, I have used Python with Django for a lot of things, and I would certainly recommend Django to anyone, due to its high secure authentication and authorization inbuilt system, a ready to use admin platform, template tags, and many more. Well, I guess that you would like to use Python to create the backend of your application, an API, and React Native for the frontend. Python and JavaScript (React) are on the trend these days and have a huge community, so there are many resources, tutorials, great documentation. I have not really heard anyone using Flutter and Go for applications these days, so I would not recommend it to you, it would make your life much more difficult.

Hope that helps, and good luck with your project!

See more
Tony Chong
Principal & Founder at Airwave Tech · | 6 upvotes · 341.6K views
Recommends
on
FlutterFlutter

I'm typically agnostic when it comes to picking languages. Whatever gets the job done, but, in this case, to figure out what's involved with what you want to do, it's going to be much more than just picking programming languages for your client and backend interfaces.

So, I'm recommending you use Flutter+Firebase as a way to figure out what you need to get done. It supports both iOS and Android out of the box, introduces you to a bunch of components you will need to think about in the future (whether you stick with Firebase or not), and the key here, is that there are tons of articles, youtube videos, and other courses you can take to pick it up pretty quickly. You could even clone an Instagram knockoff from github. Guess what else, it's all free. You might not need to worry as much about the backend since there are client libraries for Flutter/Dart for Firebase.

Some might have different opinions, and like I said, I'm usually agnostic, but in this case, you have a lot to consider. Where are you going to store the data? Are people going to need to login? Will there but customized settings the will save even if I close the app? Yeah, that's just a few questions.

Those are just a few. Lots to consider, so if you want to get something in your hand as soon as possible, try a search for flutter + firebase + chat + Instagram or something like that and have a look.

See more
Recommends
on
React NativeReact Native

If this is for learning about how to design the system, then pick the tools are you are confortable with.

Often times, I get stuck picking the tools (and trying to learn about them) vs actually trying to design the system itself.

If you are familiar with React (check out Expo) and Django then I would recommend going with that.

For deploying your backend, I would go with a provider like https://zeit.co/ that automates a whole bunch of deployment steps with their cli tools that you might have to do with AWS.

See more
Emmanuel Kayode
Software Engineer at Teamapt Ltd · | 3 upvotes · 338.5K views
Recommends
on
GolangGolang

The above listed tools will do the job, you just need to figure out your architecture(e.g models). How they will all connect. Then you can use a tool you are comfortable with to implement them.

See more
Charles Nelson
Recommends
on
PythonPython

What you need to take a look at is Apache OpenMeetings. It already does what you want, it is open source and well documented and only requires that you design the UI and plumbing required to serve you application.

See more
Adam Ha
Recommends
on
React NativeReact Native

Let's select right tool you feel you are good at. And selecting tools are used by large community to solve your stuck if encounter

See more
Needs advice
on
GolangGolangJavaScriptJavaScript
and
PythonPython

We are converting AWS Lambdas from Java due to excessive cold start times. Usage: These lambdas handle XML and JSON payloads, they use s3, API Gateway, RDS, DynamoDB, and external API's. Most of our developers are only experienced in java. These three languages (Go, Node.js, and Python) were discussed, but no consensus has been reached yet.

See more
Replies (5)
Jordan Gregory
Cloud Operations Manager at Plainsight AI · | 4 upvotes · 389.9K views
Recommends
on
GolangGolang

I've worked with all three of these languages and also with Java developers converting to these languages and far and away Go is the easier one to convert to. With the improved cold-start times and the ease of conversion for a Java developer, it is a no-brainer for me.

The hardest part of the conversion though is going to be the lack of traditional Classes so you have to be mindful of that, but Go Structs and interfaces tend to make up for what is lost there.

Full Disclosure: I'm a 95% Go convert (from Python) at this point in time.

See more
Ahmet Yildirim
Software Engineering Consultant at UXCraft Sweden AB · | 3 upvotes · 389.9K views
Recommends
on
GolangGolang

Although I am primarily a Javascript developer myself, I used Go to build AWS lambda in a similar scenario to yours. AWS libraries felt better integrated on the Go side, I believe due to the language itself (e.g. how JSON objects are handled in go). Besides that performance of Go is much superior. But on the cons side; community is far smaller around Go, compared to Javascript. That is easy notice if you look at repos of community-maintained libraries for Go. That can feel a bit unreliable.

See more
Jason Scheirer
Senior Software Engineer at EasyPost · | 2 upvotes · 389.5K views
Recommends
on
GolangGolang

Go would provide the easiest transition for Java programmers -- its IDE/tooling is second to none (just install Goland) and the deploy/distribution story is extremely clean and lends itself to work well in lambda: single, static binaries with quick startup. No need to set up a full environment or package dependencies on your lambda AMIs, just copy a file.

See more
Russel Werner
Lead Engineer at StackShare · | 1 upvotes · 389.5K views
Recommends
on
JavaScriptJavaScript
at

If you want to prioritise language familiarity, JavaScript is more like Java than the other choices; and it can be optimised to run very fast. However if you need really fast cold-start times, you can't beat Go since it's compiled. There are other things to consider, such as the massive amount of community packages and help/documentation in the JavaScript ecosystem. Go is newer but seems to be quite popular if you need something that runs fast in a single binary.

See more
Paul Whittemore
Developer and Owner at Appurist Software · | 1 upvotes · 389.5K views
Recommends
on
GolangGolang

I was initially going to suggest JavaScript due to the smaller size needs of AWS Lambdas code and the larger range of libraries and community available (and to avoid Python for this). But I have to agree with the recommendations and rationale of @ayildirim above and I think you should choose any reasonable language that is low-overhead, fast startup, and best supported by AWS Lambda, and that is probably Go. I don't think you are likely to go wrong with that, while you can potentially with the others.

So I'd agree, on the strength of AWS Lambda support and the solid performance of Go, it seems like your best choice here for Lambdas (and I'm going to need to consider that myself going forward... pardon the pun).

See more
Decisions about PySpark and Python
awesomebanana2018

1. Type safety and inferred types

Go is type safe by default, which allows you to right more reliable code and have better developer tooling, plus with the := operator, you can initialize a variable without having to define its type because it automatically gets its type from the initial value.

2. Performance

There isn't much to be said here, but on most counts go beats both Python and Node.js on performance.

3. Documentation

I'm not talking about the Go language itself, although it does have good docs. I'm talking about Go's auto generated documentation tool, which allows people to document their packages easily and works amazingly with Go's type system.

4. Compiles to binary

If you are making a local program for somebody and they don't want to download the Go compiler, you can make Go into a native binary.

5. Built for the web

Go has built in Http libraries to rival Express.js and has a HTML/Text templating system.

6. Great Concurrency

Go utilizes Goroutines to help developers utilize multiple threads easily.

Conclusion

Go is an excellent choice for any system code, especially http networking and web backends.

See more
Kyle Harrison
Web Application Developer at Fortinet · | 17 upvotes · 347.2K views

Node continues to be dominant force in the world of web apps, with it's signature async first non-blocking IO, and frankly mind bending speeds. PHP and Python are formable tools, I chose Node for the simplicity of Express as a good and performant server side API gateway platform, that works well with Angular.

See more
Octavian Irimia

Both PHP and Python are free but when it comes to web development PHP wins for sure. There is no doubt that Python is a powerful language but it is not optimal for web. PHP has issues... of course; but so does any other language.

Another reason I chose PHP is for community - it has one of the most resourceful communities from the internet and for a good reason: it evolved with the language itself.

The fact that OOP evolved so much in PHP makes me keep it for good :)

See more
Thomas Miller
Talent Co-Ordinator at Tessian · | 16 upvotes · 227.1K views

In December we successfully flipped around half a billion monthly API requests from our Ruby on Rails application to some new Python 3 applications. Our Head of Engineering has written a great article as to why we decided to transition from Ruby on Rails to Python 3! Read more about it in the link below.

See more
Ítalo Vietro
VP Platform Engineering at UrbanSportsClub · | 7 upvotes · 223.9K views

We decided to use python to write our ETLs and import them into metabase via a lambda. Before python we tried using Go, but overall go was way more verbose than Python when writing the ETLs. Go also had some issues managing memory when using the S3 upload manager library. This was a deal breaker for us that made us switch to Python.

In the end the solution was much cleaner and maintainable.

See more
Jordan Gregory
Cloud Operations Manager at Plainsight AI · | 3 upvotes · 140.5K views
Migrated
from
PythonPython
to
GolangGolang

A number of years ago; I had done python for a long time prior to learning about Go. Most of what I wrote was system-like things and web-things in python, and I got tired of running into the lack-of-a-type-system problems that python gave me. I wanted to switch to a compiled, strongly-typed system that wasn't C/C++ (been there, done that, got the "shoot yourself in the foot" t-shirt). I looked into both Rust/Go, and for what I wanted to do (system/web) stuff ... at the time, Go was the strongest candidate, so I switched and never went back. Recently I started to re-look at Rust for system things, but for anything I do that I have to touch the web with, it will be Go from now on.

See more

We needed to incorporate Big Data Framework for data stream analysis, specifically Apache Spark / Apache Storm. The three options of languages were most suitable for the job - Python, Java, Scala.

The winner was Python for the top of the class, high-performance data analysis libraries (NumPy, Pandas) written in C, quick learning curve, quick prototyping allowance, and a great connection with other future tools for machine learning as Tensorflow.

The whole code was shorter & more readable which made it easier to develop and maintain.

See more
Mike Fiedler
Enterprise Architect at Warby Parker · | 3 upvotes · 218.2K views

When I was evaluating languages to write this app in, I considered either Python or JavaScript at the time. I find Ruby very pleasant to read and write, and the Ruby community has built out a wide variety of test tools and approaches, helping e deliver better software faster. Along with Rails, and the Ruby-first Heroku support, this was an easy decision.

See more
Kyle Harrison
Web Application Developer at Fortinet · | 2 upvotes · 144K views

The decision behind choosing a server side technology is never an easy one. Every single language has it's pro's and con's around each.

For me, this decision came down to a couple simple points: 1. Node is a web tech first class citizen, designed around handling web events, in a web technology world 2. Asynchronous

The thing about Python and Java is that they TOO can handle these, and handle these very well. Java for instance powers most of Twitter and Netflix's architecture. Where Python is what's behind giants like Instagram and Patreon. Certainly, you can't go wrong. Heck, Ruby powered GitHub and GitLab, and those things see HUGE traffic.

But this project is a web technology first. And node feels right at home as it itself is a web technology. This decision was more about homogeneous synergy than most anything else. I need it to be screaming fast, asynchronous, and play extremely well with web standards.

Node fits the bill on every front.

See more
Mohamed Hassan
Software Engineer at YottaHQ Inc. · | 4 upvotes · 169.6K views

PHP is easy to learn and you can get up and running in no time, available on almost all hosting providers and you can find developers easily. It has some great frameworks for building your backend like Symfony and Laravel. However, it can be challenging when running an enterprise and needs some adjustments, very recommended for starting a new project or startup.

See more
Get Advice from developers at your company using StackShare Enterprise. Sign up for StackShare Enterprise.
Learn More
Pros of PySpark
Pros of Python
    Be the first to leave a pro
    • 1.2K
      Great libraries
    • 958
      Readable code
    • 844
      Beautiful code
    • 784
      Rapid development
    • 688
      Large community
    • 433
      Open source
    • 391
      Elegant
    • 280
      Great community
    • 272
      Object oriented
    • 217
      Dynamic typing
    • 77
      Great standard library
    • 58
      Very fast
    • 54
      Functional programming
    • 47
      Easy to learn
    • 45
      Scientific computing
    • 35
      Great documentation
    • 28
      Matlab alternative
    • 28
      Productivity
    • 28
      Easy to read
    • 23
      Simple is better than complex
    • 20
      It's the way I think
    • 19
      Imperative
    • 18
      Free
    • 18
      Very programmer and non-programmer friendly
    • 17
      Machine learning support
    • 17
      Powerfull language
    • 16
      Fast and simple
    • 14
      Scripting
    • 12
      Explicit is better than implicit
    • 11
      Ease of development
    • 10
      Clear and easy and powerfull
    • 9
      Unlimited power
    • 8
      It's lean and fun to code
    • 8
      Import antigravity
    • 7
      Python has great libraries for data processing
    • 7
      Print "life is short, use python"
    • 6
      Flat is better than nested
    • 6
      Readability counts
    • 6
      Rapid Prototyping
    • 6
      Fast coding and good for competitions
    • 6
      Now is better than never
    • 6
      There should be one-- and preferably only one --obvious
    • 6
      High Documented language
    • 6
      I love snakes
    • 6
      Although practicality beats purity
    • 6
      Great for tooling
    • 5
      Great for analytics
    • 5
      Lists, tuples, dictionaries
    • 4
      Multiple Inheritence
    • 4
      Complex is better than complicated
    • 4
      Socially engaged community
    • 4
      Easy to learn and use
    • 4
      Simple and easy to learn
    • 4
      Web scraping
    • 4
      Easy to setup and run smooth
    • 4
      Beautiful is better than ugly
    • 4
      Plotting
    • 4
      CG industry needs
    • 3
      No cruft
    • 3
      It is Very easy , simple and will you be love programmi
    • 3
      Many types of collections
    • 3
      If the implementation is easy to explain, it may be a g
    • 3
      If the implementation is hard to explain, it's a bad id
    • 3
      Special cases aren't special enough to break the rules
    • 3
      Pip install everything
    • 3
      List comprehensions
    • 3
      Generators
    • 3
      Import this
    • 2
      Flexible and easy
    • 2
      Batteries included
    • 2
      Can understand easily who are new to programming
    • 2
      Powerful language for AI
    • 2
      Should START with this but not STICK with This
    • 2
      A-to-Z
    • 2
      Because of Netflix
    • 2
      Only one way to do it
    • 2
      Better outcome
    • 2
      Good for hacking
    • 1
      Securit
    • 1
      Slow
    • 1
      Sexy af
    • 0
      Ni
    • 0
      Powerful

    Sign up to add or upvote prosMake informed product decisions

    Cons of PySpark
    Cons of Python
      Be the first to leave a con
      • 53
        Still divided between python 2 and python 3
      • 28
        Performance impact
      • 26
        Poor syntax for anonymous functions
      • 22
        GIL
      • 19
        Package management is a mess
      • 14
        Too imperative-oriented
      • 12
        Hard to understand
      • 12
        Dynamic typing
      • 12
        Very slow
      • 8
        Indentations matter a lot
      • 8
        Not everything is expression
      • 7
        Incredibly slow
      • 7
        Explicit self parameter in methods
      • 6
        Requires C functions for dynamic modules
      • 6
        Poor DSL capabilities
      • 6
        No anonymous functions
      • 5
        Fake object-oriented programming
      • 5
        Threading
      • 5
        The "lisp style" whitespaces
      • 5
        Official documentation is unclear.
      • 5
        Hard to obfuscate
      • 5
        Circular import
      • 4
        Lack of Syntax Sugar leads to "the pyramid of doom"
      • 4
        The benevolent-dictator-for-life quit
      • 4
        Not suitable for autocomplete
      • 2
        Meta classes
      • 1
        Training wheels (forced indentation)

      Sign up to add or upvote consMake informed product decisions

      - No public GitHub repository available -

      What is PySpark?

      It is the collaboration of Apache Spark and Python. it is a Python API for Spark that lets you harness the simplicity of Python and the power of Apache Spark in order to tame Big Data.

      What is Python?

      Python is a general purpose programming language created by Guido Van Rossum. Python is most praised for its elegant syntax and readable code, if you are just beginning your programming career python suits you best.

      Need advice about which tool to choose?Ask the StackShare community!

      What companies use PySpark?
      What companies use Python?
      See which teams inside your own company are using PySpark or Python.
      Sign up for StackShare EnterpriseLearn More

      Sign up to get full access to all the companiesMake informed product decisions

      What tools integrate with PySpark?
      What tools integrate with Python?

      Sign up to get full access to all the tool integrationsMake informed product decisions

      Blog Posts

      TensorFlowPySpark+2
      1
      722
      Sep 29 2020 at 7:36PM

      WorkOS

      PythonSlackG Suite+17
      6
      3040
      PythonDockerKubernetes+7
      3
      1096
      PythonDockerKubernetes+14
      12
      2597
      Oct 3 2019 at 7:13PM

      Ably Realtime

      JavaScriptPythonNode.js+8
      5
      3821
      What are some alternatives to PySpark and Python?
      Scala
      Scala is an acronym for “Scalable Language”. This means that Scala grows with you. You can play with it by typing one-line expressions and observing the results. But you can also rely on it for large mission critical systems, as many companies, including Twitter, LinkedIn, or Intel do. To some, Scala feels like a scripting language. Its syntax is concise and low ceremony; its types get out of the way because the compiler can infer them.
      Apache Spark
      Spark is a fast and general processing engine compatible with Hadoop data. It can run in Hadoop clusters through YARN or Spark's standalone mode, and it can process data in HDFS, HBase, Cassandra, Hive, and any Hadoop InputFormat. It is designed to perform both batch processing (similar to MapReduce) and new workloads like streaming, interactive queries, and machine learning.
      Pandas
      Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more.
      Hadoop
      The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale up from single servers to thousands of machines, each offering local computation and storage.
      PyTorch
      PyTorch is not a Python binding into a monolothic C++ framework. It is built to be deeply integrated into Python. You can use it naturally like you would use numpy / scipy / scikit-learn etc.
      See all alternatives