No space left on device, for deploying Pytorch Model

[m-ali-awan]

Hi there, hope you are fine.

I am trying to deploy VQGAN+CLIP model using Sagemaker, and I am using PyTorch.
Here is my code:

model = PyTorchModel(
    model_data=model_location,
    source_dir="inference-code",
    entry_point="inference.py",
    role=role,
    framework_version="1.4.0",
    py_version="py3",
    #predictor_cls=ImagePredictor
    )
predictor = model.deploy(initial_instance_count=1, instance_type='ml.g4dn.xlarge')

Now, I am facing following error:

OSError: [Errno 28] No space left on device

Here is my requirements.txt:

torch==1.8.1
torchvision==0.9.1
torchaudio==0.8.1
ftfy
regex
tqdm
omegaconf 
pytorch-lightning 
IPython 
kornia 
imageio 
imageio-ffmpeg 
einops 
torch_optimizer

Now, I think most of the space is taken by these installations, also the sizes of models is somewhat around 2 gbs. Now once I try local mode. Then I see that my following directory space runs out.

the "/dev/nvme0n1p1", is 100 % utilized, and 'out of space error' is displayed.

So, kindly help me with, making it run on local mode, for this I think I need to change the directory where these installations are done.

Also, I think there are some container size limits, that's why it is giving me out of space error.

Thanks, for any help.

[prameshbajra]

Hey @m-ali-awan ,
Sorry, this is not an answer but I am facing the same issue when trying to deploy my model to a serverless endpoint.

I am doing this currently.

deployed_result = model.deploy(
    endpoint_name = endpoint_name,
    instance_type = 'ml.g4dn.xlarge', 
    initial_instance_count = 1,
    serverless_inference_config= ServerlessInferenceConfig(max_concurrency=5, memory_size_in_mb=6144),
    serializer = JSONSerializer(),
    deserializer = JSONDeserializer()
)

Did you find a way to make that work?
Thank you.