The language seems super niche to data processing and Spark. The learning curve is steep and I was wondering if it's worth investing time into it for future career prospects.
Why do people say the learning curve is steep? It seems straight forward to me, and pretty dang cool actually.
Yes its not only Spark. Scala is the pioneer in functional programming and you can use even Java APIs in it. Even Akka Framework which is defacto now in Reactive programming is written and best used with Scala. Most of the important technologies, Spark, Akka, Kafka etc are written in Scala so its good to understand. I studied Scala myself. Syntax is bit wierd in the beginning but you'll soon get a hang of it.
I'm currently a data engineer attempting to learn Scala for setting up streaming data pipelines for my team. Can the Akka framework help further in my work or is that a completely different usecase
I’m extremely biased since most of our codebase is in Scala, but I do think it’s worth it. I have a Go, C++, Python background and the more Scala I write the more I am impressed by how little code I need to write to get things done. Syntax is definitely a bit jarring in the beginning (pun intended) but you’ll pick it up in no time. I highly recommend Hands-On Scala Programming by Li Haoyi (free sample PDF on his website) and Scala Cookbook (O’Reilly) to get started.
It's a good language for both OOP and functional programming, but your current and future paths may not benefit much from it. Data engineering with spark is great in Scala, but Python and SQL are far more common now. Primitives in Scala of course. Backend services are also in Scala sometimes as mentioned above. What are you trying to do and why? That would help reach a decision
Thanks for your reply. My new team requires some amount of Scala knowledge. We're currently building out capabilities to feed reL time data into our models. I was thinking of specializing Spark Scala to build a marketable skillset. That's why I asked this question.
Actually wanted to say just that. What is better marketable is what is niche and difficult. I had experiences in C, Objective C, Javascript, Java etc before starting with Scala. And yes its difficult and very few know it. Then for ML I learnt Python and it was a breeze. Probably Python is the easiest language out there.
Scala is a cool language worth learning if you have to opportunity to use it. Functional programming is incredibly expressive and powerful once you wrap your head around the concepts.
Most of the times, people/teams who want to use scala dont really know how to “use it”. I agree with what databricks/jp mc folks have said below. If your team or an org or the company itself does nit have folks who have previous experience in working with say reactive programming, or functional programming, good luck keeping up the morale high. Scala comes in with its type inference, and functional programming features. The compiler does a “lottt” of stiff under the hood(the build tool sbt fucking sucks and i hate it). So i would say learn Scala to see what it takes to write a purely functional code, also learn tail recursion(see the book functional programming principles in Scala for those exercises). Then try to understand the play framework, and other systems the folks have mentioned below.
Doesn’t Java have functional programming features now? Why would you learn scala when you can code in Java
Absolutely
Scala version 3 fixes many of the complaints with earlier versions.
Learn it when it comes.
Agreed. There are too many things to learn. Learn if you need to use it.
Gotcha. I'm currently needing to learn Scala to work on my team's codebase so I guess it's time lol