Baseten 推出自动扩缩容计划，提前扩容降低冷启动延迟
Baseten 推出的自动扩缩容计划允许用户在预测流量到来前提前增加副本数，从而有效降低冷启动延迟。用户可设置重复或单次运行的计划，每个环境最多配置 10 个时间窗口。相关配置可通过仪表盘、CLI 或 Management API 进行设置。
TL;DR
Autoscaling schedules reduce cold-start delays by raising the minimum number of replicas before predictable traffic arrives. You can set a schedule to recur (e.g., weekday mornings) or to run once (e.g., ahead of a product launch). Configure up to 10 schedules per environment through the dashboard, CLI, or Management API.
Autoscaling adjusts the number of replicas to match incoming traffic. When demand increases, new replicas need time before they can handle requests. This is their cold-start time, which includes allocating compute resources, starting the serving software, and loading the model. Requests that wait during a cold start have longer end-to-end response times.
Autoscaling schedules help avoid cold-start delays by starting additional replicas before traffic arrives. You can set a schedule to recur (e.g., every weekday morning) or to run once (e.g., ahead of a launch).
Schedule autoscaling around demand
Many workloads have predictable traffic. Autoscaling schedules let you change your autoscaling settings at set times, so you don’t need to maintain an external cron job to do it.
Recurring peaks: Raise the minimum number of replicas during work hours and lower it overnight. For example, a meeting-transcription app can scale up before meetings begin and back down after the peak.
Planned events: Schedule a temporary capacity increase for product launches, load tests, or migrations to a new deployment.
✕
Illustrative example: starting additional replicas before an expected traffic surge can help keep end-to-end response times consistent as request volume increases.
How autoscaling schedules work
Each schedule sets a time window and the minimum and maximum number of replicas for that window. The minimum sets the lowest number of replicas the autoscaler keeps running during the window. The maximum limits the total number of replicas the autoscaler can run. When no schedule is active, the environment uses its default settings.
To configure a schedule:
Create up to 10 schedules per environment, all using one shared timezone.
Choose a daily, hourly, or one-time window. Windows can’t overlap and must last at least five minutes.
Use the dashboard, CLI, or Management API.
✕
Example from the Baseten UI: Two autoscaling schedules for a production environment.
Get started
Start replicas before your expected traffic arrives to reduce cold-start delays. Follow the setup guide to configure schedules through the dashboard, CLI, or Management API, or talk with our engineers to customize the setup for your workload.
来源：Baseten 工程博客 · baseten.co