+1. 6.5 YOE, primarily Java backend developer. Solid experience with complex multi threading servers, and a great depth of understanding in using the best distributed system for specific tasks (Cassandra, dynamodb, Kafka, flink, spark, temporal)
How to learn about multithreading with implementation ?
Also currently il working Kafka and Spark - can you recommend some good resources to learn Spark ?
Operating Systems class will give you a great basis. Then writing/reading about rust, Java, go, and even JavaScript will give you a great variety of understanding of different cooperative/parallelization models. Try building the same service using many different paradigms. For example I recently evaluated using Java VirtualThreads (aka green threads similar to goroutines) for a Cron job, but we actually achieved better utilization and less context switching by using a thread per core model to parallelization the batch reads and then spawning a single threaded event loop (via Netty) on each core for concurrently updating a Dynamo table within each thread.
Similarly with the distributed systems softwares, a distributed systems class will give you great foundations and then you can build on those by using multiple systems. At Amazon our scale was so large we had to use dynamo. I’ve been lucky enough to work with that, Cassandra, Kafka, flink, and temporal for different use cases. Kafka is great for the pub sub model to maximize resource utilization. Temporal is like a more generalized MapReduce job that has a lot of different use cases.
In summary, get a foundational understanding then apply to jobs where you’ll get to use these technologies.
+1. 6.5 YOE, primarily Java backend developer. Solid experience with complex multi threading servers, and a great depth of understanding in using the best distributed system for specific tasks (Cassandra, dynamodb, Kafka, flink, spark, temporal)
How to learn about multithreading with implementation ? Also currently il working Kafka and Spark - can you recommend some good resources to learn Spark ?
Operating Systems class will give you a great basis. Then writing/reading about rust, Java, go, and even JavaScript will give you a great variety of understanding of different cooperative/parallelization models. Try building the same service using many different paradigms. For example I recently evaluated using Java VirtualThreads (aka green threads similar to goroutines) for a Cron job, but we actually achieved better utilization and less context switching by using a thread per core model to parallelization the batch reads and then spawning a single threaded event loop (via Netty) on each core for concurrently updating a Dynamo table within each thread. Similarly with the distributed systems softwares, a distributed systems class will give you great foundations and then you can build on those by using multiple systems. At Amazon our scale was so large we had to use dynamo. I’ve been lucky enough to work with that, Cassandra, Kafka, flink, and temporal for different use cases. Kafka is great for the pub sub model to maximize resource utilization. Temporal is like a more generalized MapReduce job that has a lot of different use cases. In summary, get a foundational understanding then apply to jobs where you’ll get to use these technologies.