hi to all blinders im very newbie and wonder how to reduce the gpu memory usage(personal curiosity). is there any recommended way ? i read about batch chunck what FAIR introduced but it seems it only gives the opportunity to train efficiciently not for deployment or actually usage.
Use CPU instance instead
Use GCP or Azure instead
Use auto scaling, use less instances when itโs not being used
Use the instance in a region thatโs cheaper
Use dedicated instances if you are planning to use it for long time
Use spit instances if you are ok with your instance being taken away anytime
If you waste a lot of GPU time while communicating over the network, use a network optimized GPU instance (P3n)
Hey thanks foe the response! Other than instances , any paper or methodology?
Is there any methodology or theory or paper?? Just want to learn new way!
Did you try everything said so far?
Terminate the instance and release