This week covered "train with my code, scale training, look inside training, evaluate results correctly"—the second half of model development. Moving beyond built-in algorithms into custom training expands choices; judgments of "how much freedom vs how much managed services" become critical. Today we re-weave four topics into one flow.
[Custom Training] How far to bring my code
Script mode → requirements extension → BYOC
[Distributed Training] What to split
Data parallel (slow) ↔ Model parallel (doesn't fit)
[Debugging·Profiling] Look inside training
Debugger (model tensors) ↔ Profiler (resources)