In most cases, you can simply set backend='zentorch' as an argument in torch.compile() to enable optimizations.
For Hugging Face large language models, up to r5.1 we used to provide zentorch.llm.optimize(), a specialized method that
delivers further performance enhancements.
However, beginning with r5.2, Zentorch.llm.optimize has been removed. LLMs are run with zentorch as
a plugin in vLLM. Refer to the vLLM-zentorch Plugin
section for details.
For additional guidance on usage scenarios, refer to the Recommendations section.
import torch
import zentorch
from torchvision import models
model = models.__dict__['resnet50'](pretrained=True).eval()
compiled_model = torch.compile(model, backend='zentorch', dynamic = False)
with torch.no_grad():
output = compiled_model(input)