NewyaRp58

Deploying CNN interview question

Had an interview question about how to deploy a CNN. Resume had no mention of CNN, DL or any tool kits like PyTorch, TF, etc. Background is in streaming (Flink, Spark Streaming) where I deploy ML models from MLLib in Spark Streaming and work with JVM Pojo objects / binaries in Flink. After I open up the ports and ID paths, write the apis to send / receive requests, write pipelines for the data to be sourced from Kafka queues, wire transmissions using grpc, load balancing through Nginx, service manager as Zookeeper, Groovy my way through build pipelines, artifact deployment by way of resoucing from Artifactory, all in an multi-cluster, on-prem environment. When I explained this to the interviewer, the only feedback was there is a disconnect, no other response. I don't think the interviewer was concerned about QPS or other metrics since we didn't get that far. I feel like he may have been looking for an obvious answer, but can't quite identify what he was maybe looking for online. Can someone shed light on what maybe was being asked and high level what would be a suitable response? Want to learn more, should be studying DL Ops now I guess because I feel like I've got a handle on ML Ops, but I'm drawing a blank on the DL side after this interview. #datascience #DL #dataengineerinterview

Amazon Pintxos Apr 26, 2021

With which company? They were probably trying to gauge high-level approach to systems design.

Capital One HdYF70 Apr 26, 2021

Not really sure what they’re looking for. Usually isn’t a big deal to deploy a model, even if it’s a CNN. Maybe they were concerned about the size of the model? Aside from that, just seems like API calls to use the model for predictions. Then there’s the nuances like persisting the model for performance, but a model is a model. Doesn’t make it special if it’s a CNN. Someone please correct me if I’m wrong. I’d love to learn a bit about this area as well.

Deloitte PALd11 Apr 26, 2021

They might be interested in how they scale it out and etc... Not all that trivial.

New
yaRp58 OP Apr 26, 2021

@PALd11 Can you shed a few sentences about scaling it out? I use multiple models of the same version hosted throughout our environment and the load balancer is configured to use a round robin mechanism to distribute the workload within the clusters among the separate hosts. Maybe this is an out dated way of doing things???, would love to get insight on how scaling out is done with CNNs. I'm probably not too far off just need some updating ...