I did 8.5 security engineer on-sites with top tech companies…a prep guide

See where the 0.5 on-site came from later... I’m writing this to share my experience with my recent job search in security. There is tons of information for SWEs but not much for SEs and I’m tired of the weekly threads under “Security Career” asking “what to expect for interview at company X (especially Google)” with like no response except responses of "any updates, OP?" from sad pleb moochers. Of course OP never responds. Also, I think security professionals need to be informed on what the compensation possibilities are in tech. There are still people running around here not knowing what security folks could be making, including recruiters. Maybe if I spread this info, the interview process would be more standardized and the comp as well so I won’t be grieved with low-ball offers and the excuse "it's not fair if we pay you more because the existing staff engineers that live in LCOL places at this company don't even make what you're asking for and you're less senior". As it stands, security engineer interviews and compensation are the wild west (read on and you’ll see). I’ve been studying and applying to jobs since beginning of February 2022. It’s now almost end of June, so this whole process took 5 months. I did many recruiter calls, phone screens, and landed 8.5 on-sites, 2 offers. I have 8 yoe, 1.5 as a dev and 6.5 in security. I have a diverse work experience and I worked for a relatively prestigious company (the job after Oracle) so I had no trouble getting my resume noticed by recruiters and getting interviews. Most of these companies reached out to me, I didn't even apply to them. I don’t specialize in any particular field because I wanted to be a well-rounded security professional with T-shaped experience, focusing on breadth first, then depth. I planned on exploring different areas before I dove deep into a specialization. For my next job, I wanted to pivot into joining a blue team (I have no such experience). However, due to my dynamic exp, I found it difficult to convince every recruiter to let me interview for a role of my choosing. It all depended on what roles were open at a company, how competent the sourcer was at matching the role with my experience, and how much the hiring manager wanted to give me a chance. As a result, I interviewed for detection and incident response, appsec, vulnerability management, and corporate security roles. I was leveled at L4/L5 roles for all companies based on initial phone screen. Here are the companies I talked with. I will not say which role I interviewed for at which company (except for Amazon since I interviewed for 3, lol). General recruiter call but no interview (due to lack of fit for an open role): —RH —Stripe Phone screen but no onsite: —Plaid (I count plaid as 0.5 onsite because it was 3 rounds instead of 1 call) —AWS (SIRT) —Amazon (detection and response) I failed the SIRT interview with flying colors and was told I couldn't interview anymore for 12 months, but I cheated the system and applied to another Amazon job with a new email and no one knew. —Asana (cancelled on-site due to hiring freeze) Virtual onsite: —Datadog —Pinterest —Google —Zoom —Uber —Meta —Snap —Amazon (appsec) Offers (look at difference lol): Zoom ZP3 - ~260k TC, ~20k sign-on Snapchat L4 - 470k+ TC (took this offer) Amazon (appsec) - interviewed for L5 but given L4 offer. Recruiter told me the max pay for L4 is 205k base, 295k TC for NYC. Didn’t bother talking further. What were the interviews like? ======================== Depending on the company, I would break the rounds into various types 1. security domain knowledge with open ended questions and some trivia, 2. threat modeling round, 3. code review + web app trivia, 4. coding/scripting, 5. system design, 6. past project/system architecture walk through 7. behavioral/leadership/cultural fit. It was a PAIN and INCREDIBLY stressful to prepare for all these rounds. Security engineers have to work harder than SWEs because SWEs only have to study coding, system design, behavioral, and maybe the system architecture walk through (substitute for system design). With practice and once they do enough Leetcode, see enough system design archetypes, they can just apply the same concepts to every single interview. They are all cookie cutter, pretty much the same. That’s how SWEs can land like 5 offers at a time. It’s much more difficult to practice and prepare for the security domain interviews as security engineers because the field is so broad. We also don’t have a question bank like Leetcode or Grokking The System Design course. 1. Security Domain Round: ======================== I was tested for breadth and depth of knowledge and grilled on MANY questions in all layers of the OSI model. I’m not going to say which company I was asked which questions. Honestly, all of them are fair game for any of these companies. Some examples I can recall from memory: How do you analyze a suspicious email attachment? How do you attack VPN? What happens when you visit google.com in browser? (classic one) How do you find vulnerable software running on people’s work machines? You visit a team website and get a 404 but everyone else on your team can access it, what is wrong? A CEO of a well known company is sitting across from you at a coffee shop how do you hack into his email? Your manager wants you to bring your work laptop to Defcon, how do you secure it from getting hacked? What advice would you give engineers who want to design a secret management service like Kubernetes Vault? An engineer comes and tells you his computer is slow, what do you do? You find out it’s a coin miner turned ransomware. How do you stop it from spreading on your network? What is ARP poisoning and how do you mitigate it? If you were an attacker who compromised a CA and stole the private key, how would you use it? How would you attack the OAuth workflow? Explain SSL handshake. How do certificates get provisioned? You see high volume of traffic to 169.254.169.254, what do you suspect? (guess which company asked this one) What are different types of XSS and how do you prevent it? Security in-domain questions are very difficult to study/prepare for because there are so many topics and the study resources are not standardized. If you pick the wrong resource, you’re SOL and you waste a bunch of time reading and memorizing the wrong bullshit. This link has been shared around on Blind for like forever: https://github.com/gracenolan/Notes/blob/master/interview-study-notes-for-security-engineering.md. Don’t be like OMG that’s too much wtf, like me, and ignore it and tell yourself “interviews can't be that hard, I’m expected to know all that?” Yes, the interviews are that hard now. You really DO need to know most things on this list. Literally pick the domains you think are relevant to the position you are applying for and be able to talk for at least 5 minutes on any topic under that domain. For security domain interviews, you must answer everything with 90% percent accuracy or you will not get the job or you will get down leveled like me. I literally had to read and memorize whole books and articles on SSL, PKI, web security, secure system design, and threat hunting. I wish I had more time to read more stuff but at some point I got tired of prepping and wanted feedback. Maybe it’s because I was applying for senior positions and expectations are higher but https://danielmiessler.com/study/infosec_interview_questions/ and questions scraped from Glassdoor, or GitHub.com/tadwhitaker/Security_Engineer_Interview_Questions don’t cut it anymore! Do not use these lists. You will be woefully unprepared except for maybe Amazon phone screens. Only Amazon and Datadog asked elementary trivia questions like what’s the difference between asymmetric and symmetric encryption, what’s the difference between TCP and UDP, what’s the diff between encoding, encryption, and hashing etc. The rest of the questions were all open ended and very difficult to answer without having seen the situation before either from reading or from your previous jobs. 2. Threat Modeling/3. Code review ======================== I was given an architecture diagram for a doctor/patient messaging website like ZocDoc and was asked to threat model and just talk about it based on the diagram. Was asked to do a code review to find as many bugs as possible on some vulnerable java and JavaScript code and had to name the class of vulnerability it was. I was asked to suggest improvements to an existing API (redesign the inputs for example). 4. Coding/Scripting ======================== Typical coding problems (I did all in python) not including those asked by Uber and Plaid (which were LC hard math problems): —Parse timestamp and IP from apache log files —Correlate data from 2 separate logs, parse and store them using hashmap of lists so you can look it up in O(1) time later —Count frequency of words that appear in a text —Make API calls using requests and write tests for your methods —Write a caesar cipher —Leetcode easy string manipulation questions —Leetcode medium string problems involving a stack 5. System Design ======================== I was asked to design: —Malware scanning reporting dashboard like Virus Total —Web crawler using a botnet —Auction website like Ebay (literally just a regular SWE round, no security asked) 6. Architecture walk-through ======================== I had to pick a project I worked on in the past and talk about it. What were the parts. How did I handle scaling? This was a difficult task because I only had 1 job, my job at Oracle, where I worked on a system that was large enough scale to talk about this. 7. Behavioral Round ======================== Typical “Tell me about a time when…” and you have to answer using STAR format. Just do an on-site with Amazon LPs and you'll know. Some examples: Tell me about a time when... You overcame challenges in your work You realized you couldn't deliver on something you committed to You had to work with a difficult coworker You made a bad decision and what would you do differently You took on an assignment outside of your normal responsibilities You delivered something that exceeded expectations You disagreed with someone but was able to move forward with the disagreement You received harsh critical feedback Reason for why I didn’t get offers: —Uber and Plaid literally asked leetcode hard questions, which I could not find the optimal solution to in an hour. They had a high coding bar. These two companies hard rejected me. Uber had the audacity to say there is no position for me in their security org because they require every security engineer to code 80% of the time and I didn’t want to do that. They told me I may be a fit for the IT org like Salesforce admin roles because I had SaaS experience but there was no headcount so I can’t interview. I felt really insulted. I wasn’t going to throw my security career away for an IT admin role even if they had head count. Plaid also was incredibly unprofessional and expected candidates to bring their own IDEs instead of paying for a Coderpad subscription. I didn’t read the interview memo carefully and went unprepared, had to scramble during interview to sign up for free coderpad, and the SWE interviewer dinged me for it. Snap was supposed to have a high coding bar but not for the team I interviewed with so I lucked out! —For Pinterest and Meta, I was close. Other candidates simply did better than me in the technical security domain questions critical to the job and showed more in-depth knowledge compared to me. I also did iffy on Meta system design. Competition/talent pool is high these days due to remote workers (tons of talent from midwest and from DC) and all security teams pretty much having the culture of being okay with remote work. If there wasn’t so much competition, I would have gotten the job offer. This was not the case 4 years ago where if I were invited to on-site I would often get the job. The bar for passing phone screens was higher before and companies had to invest in physically flying candidates to locations for on-sites. —After doing like 9 rounds of interviews with them, Google offered L3 down-level due to mixed feedback and team matching in a few months when more L3 positions open up. I told them to go pound sand. Datadog did the same to me but they decided to hard reject instead of down level since the team also had mixed feedback. —In general, for domain specific interviews, the interviewers were looking for a specific answer based on their own job experience. They simply didn’t like my approach/answer or asked gate keeping trivia questions you could google in 5 seconds. The younger interviewers (1-5 yoe) expect perfection these days in your answers and will say no hire if you are more senior than them in yoe but cannot answer their trivia question. I was dinged for not knowing what A stood for in CIA of infosec and didn't know the 5 steps of an incident response plan. Honestly, since interviews are virtual these days, I bet a lot of other candidates have interview notes with answers already written and can cheat off their screen or they are just very good at googling answers on the fly. Why I think I ended up getting the offers I did... I think it was 50% prep/memorization but the other 50% is complete luck—whether the interviewers and hiring manager liked me and found me personable. I found if the hiring manager and at least 2 engineers on the direct hiring team like me (for dumb reasons like used to live in same city, have a dog, have same sense of humor, but also provided I don’t completely fuck up the round) I will get the offer. The problem is lots of these interviews are impersonal, some interviewers don’t give a f*** and are strictly deadpan, and I don’t get to showcase my personality at all (for example Amazon interviews are strictly business). I found female interviewers also tend to be strictly business and graded me solely based on technical correctness. I found myself least likely to pass interview loops where the interviewers don’t work on the hiring team or where I never even meet the hiring manager (looking at you Google and Meta). In other words, if I have a 5 round interview and I only meet 2 members on the direct team, I have lower chance of passing the loop because I must impress at least 3 on the direct team and those 2 rounds with the team members are critical. If I impress the guy conducting the coding interview who's on a different team but not the person on the direct team in that critical round, I won’t pass. The feedback of the people on the direct hiring team hold more weight than that of any other interviewer. The direct team members usually conduct the domain knowledge rounds and as I’ve said above, those are a b**** to study for. Also if you get a SWE for your coding interview or system design, you are more likely to bomb that round because they won’t make concessions for you for being an SE candidate. SWEs don’t seem to have empathy for security folks’ being weaker in those areas. How a team conducts the interview is entirely not up to your control so I would say unless you specialize in applying and studying for jobs in one security domain only, interviewing for a security engineer position is a PITA! Keep practicing and just fail interviews like me, use interviews for one company as recon for the next and adapt. Eventually, you’ll have studied almost everything from learning from your knowledge gaps and you’ll get something. There is only so much interviewers can ask you. Offer TC progression (not counting refreshers/stock growth): 86k => 140k => 280k => now 470k+, 6.5 yoe (in security) #cybersecurity #interview #security

Google

Google