What skills should I develop to apply successfully for core HFT trading companies. Besides leetcoding. If you answer networking, be more specific
less traditional databases more low-latency optimizations closer to the metal: OS, hardware, networking
HFT dev is very niche. Even at Hedge Funds that is less than 5 percent of all devs. And the way C is written at Hedge Funds is something you may not adjust to. But if you really want to start trying to code close to bare metal. Reddit has some good discussions around this. Also a lot of work is in OpenMPI and OpenCL and you know your hardware and its instruction set that you will program to and the cache coherence when accessing memory. If NUMA means a clothing brand to you then you won’t like it.
Linux kernel internals dealing with concurrency, process scheduling, I/O handling, networking, memory allocators. Kernel bypass for userspace processing of packets. Computer Architecture- Pipelining, cache coherence, branching, vectorization. C++ - metaprogramming with templates and constexpr, CRTP to allow static polymorphism instead of dynamic, reducing virtual functions so that vtable lookups are reduced, RVO/NRVO, move semantics, reduced if statements to reduce branching and jumps, expression templates, placement new and using preallocated memory pools to avoid constantly using the memory allocator, cache friendly code using performant STL containers like vector over non performing ones like map, unordered map, avoid shared_ptr since reference counting is an expensive operation and use unique_ptr only when needed, reduce use of C++ exceptions. Userspace level in Linux you should know non blocking I/O , Interprocess Communication, shared memory. Resources- 1) Linux Kernel Development, Robert Love 2) Linux Programming Interface, Michael Kerrisk 3) Understanding Linux Network Internals, Christian Benvenuti 4) Richard Stevens TCP/IP books as a reference since it is 3 volumes. 5) C++ Templates 2nd Edition by David Vandervoode, 6) C++ Concurrency in Action 2nd Edition by Anthony Williams 7) Modern Effective C++ by Scott Meyers 8) Andrei Alexandrescu’s C++ book 9) r/cpp on Reddit 10) Sergey Zubkov’s answers on Quora 11) Computer Systems, A Programmers Perspective, 3rd edition. Chapters 4,5,6,8,9 are important. 12) Agner Fog’s C++ Optimization guide, freely available on his website.
I'm trying for HFT SWE, are these resources (still) applicable for this role?
No, now a supercool “hft.h” is available using which you can skip everything listed above.
But does it really pay well as HFT dev? I assume there is a lot of stress that comes with it, simple bug could cost millions, just guessing
We're always on the look out for C++ Software Engineers who can go in-depth on the following topics for roles in New York, Chicago, Austin, Sydney, Singapore, London, Paris. You should prep on runtimes [searching, sorting, priority queue, hashmap, binary search]. Also do a deep technical discussion on trade offs and internals. Networking • Handle network errors and exceptions in applications • TCP, Multicast, UDP differences, use cases. • Key steps in creating a basic TCP server. • Asynchronous networking, C++ libraries. • Itch, Ouch, Fix. • TCP congestion control. • Handling jitter Serialization/Deserialization • Optimize serialization and deserialization for performance. • Text-based, binary-based serialization formats. • Data format in serialization, deserialization context. • Handle versioning, backward compatibility in serialized data. • Handling edge cases like malformed data. • Error Handling and Recovery • Deserialization Overhead: • Peak Load Handling during flash crashes, VIX spikes Buffer overflows • Mitigation techniques • impact on network performance? • Monitor, quantify buffer overflows impact on system performance Virtual memory • Operating system handling page faults in virtual memory. • Describe paging, its use in virtual memory. • Explain demand paging in virtual memory. • Explain page swapping in virtual memory. • Translation Lookaside Buffer (TLB) role in virtual memory. • System decisions on pages in physical memory. • Page replacement algorithms concept explanation. • Caching, Cache-efficient algorithms, L1, L2, L3. • Direct-mapped cache, Set associative cache. • Taxonomy of cache misses, Cache analysis of matrix multiplication. • Tiled matrix multiplication, Divide and conquer. • Cache-oblivious algorithms, FIFO advantages & limitations. • Potential drawbacks of MRU, Belady's Algorithm. • Write-through, write-behind cache management strategies - Trade-offs. • Block vs associative caches • Cache prefetching internals. Multithreading • Avoid synchronization issues like race conditions, deadlocks? • Blocking I/O vs non-blocking I/O internals, use cases, tradeoffs. • Starvation, fairness schemes, priority scheduling. • Coffman conditions: Mutual Exclusion, Hold and Wait, No Preemption, Circular Wait. • Discussion on debugging non-determinism, synchronization problems, concurrency issues. • map vs unordered_map • auto_ptr vs smart ptr • Benchmark performance code and NUMA • Spin lock • "const int*" and "int const*"? Memory Management • Strategies for detecting, preventing memory leaks. • Custom memory allocators for specific requirements. • Memory-profiling tools for usage patterns, potential issues. • Stack and heap memory allocation internals. • new and delete operators, usage in dynamic memory allocation. • RAII and its role in managing resources. • Smart pointers: unique_ptr, shared_ptr, weak_ptr usage. • How smart pointers prevent memory leaks, dangling pointers. • Reference counting and its application in smart pointers. • Performance implications of using smart pointers vs. raw pointers. • Internals of memory pools.
hey thanks for the incredible writeup! Which company would that be?
Lock free data structures, kernel bypass for network, OSI model, write SQL query, concurrency, condition variables, locks, mutex, OOP.