Developers of artificial intelligence can now deploy and scale their workloads using an expanded set of infrastructure tools on the Modal platform. The general availability of multi-node clusters allows for the training of models with as many as 1T parameters, a scale already achieved by user Decagon. These clusters provide RDMA-connected nodes that are accessible instantly through a single decorator using the rdma=True configuration.

The updates to Modal's Runtime also introduce VM sandboxes, sandbox sidecars, and sticky sessions to improve workload isolation and request routing. These tools enable high-performance distributed computing by providing developers with more control over how compute nodes are instantiated and managed.