用 Semaphore 控制并发数
基本写法
sem = asyncio.Semaphore(10)
async def fetch(url):
async with sem:
return await client.get(url)
await asyncio.gather(*(fetch(u) for u in urls))
配合队列还能做更精细的背压控制。
sem = asyncio.Semaphore(10)
async def fetch(url):
async with sem:
return await client.get(url)
await asyncio.gather(*(fetch(u) for u in urls))
配合队列还能做更精细的背压控制。