机器学习推理工程师 – Terminal-Bench 任务设计师(vLLM / SGLang / TensorRT-LLM)
ML Inference Engineer â Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)
全球可投
设计
预计有效期:2026-12-20
🏢 公司 / 雇主
Toptal
🛠 技能要求
PyTorchJAXvLLMSGLangTensorRT-LLMPython命令行DockerGitGPU 性能分析自动化测试
📋 职位要求
- 设计专家级 Terminal-Bench 任务,将你亲身经历的真实有偿机器学习推理工作转化为 AI 智能体必须端到端解决的挑战。 Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end.
- 定义每个任务所运行的文件、系统和环境(例如带流式输出的模型服务),确保挑战真实、可复现且可通过命令行解决。 Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable.
- 为每个任务编写可运行的参考解决方案,并验证其可端到端运行,工作量按约 60 至 75 分钟的专家投入设计。 Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort.
- 构建自动化检查,客观证明结果的正确性,包括机器学习输出静默出错的情形,例如数值漂移或指标错误。 Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs.
- 使用 Docker 和 Git 打包任务以保证可复现性,并记录所涉及的系统、工具、数据和规则,以便审阅者验证难度。 Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty.
- 3 年以上亲身从事机器学习推理或服务工作的经验(而非仅学习、销售或管理该领域),或拥有博士学位并在该细分领域有活跃研究 3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield
- 在过去 3 年内从事过此类工作,并能列举所使用的系统、工具、数据和规则 Did this work in the last 3 years and can name the systems, tools, data and rules used
- 曾在真实规模(多 GPU 或生产环境服务)下训练、部署或评估模型,而非仅使用笔记本或 API 调用 Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls
- 熟悉 PyTorch 或 JAX 内部机制,以及 vLLM、SGLang 或 TensorRT-LLM 等服务技术栈 Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM
- 能够解释机器学习结果在无报错情况下出错的原因(数值漂移、指标错误)以及如何发现这些问题 Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them
- 熟练掌握 Python 和命令行,以及文件、CSV/JSON、API 或数据库操作 Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases
- 能在 2026 年 11 月初开始工作,并能完成每个约 60 至 75 分钟的任务 Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each
📄 职位描述(中文)
概述
我们正在构建一个基准测试,用于检验 AI 智能体在终端中处理真实专家工作的能力。我们正在招聘机器学习推理与服务领域的专家,将你们工作中真实的有偿任务转化为专家级的 Terminal-Bench 挑战。每个任务包含运行环境、可运行的解决方案,以及证明结果正确的自动化检查。***本职位采用按任务计酬模式。详情请查看申请问题部分。***
职责
• 设计专家级 Terminal-Bench 任务,将你亲身经历的真实有偿机器学习推理工作转化为 AI 智能体必须端到端解决的挑战。
• 定义每个任务所运行的文件、系统和环境(例如带流式输出的模型服务),确保挑战真实、可复现且可通过命令行解决。
• 为每个任务编写可运行的参考解决方案,并验证其可端到端运行,工作量按约 60 至 75 分钟的专家投入设计。
• 构建自动化检查,客观证明结果的正确性,包括机器学习输出静默出错的情形,例如数值漂移或指标错误。
• 使用 Docker 和 Git 打包任务以保证可复现性,并记录所涉及的系统、工具、数据和规则,以便审阅者验证难度。
• 根据审阅者反馈和智能体测试运行结果对任务进行迭代,调整难度并消除歧义,确保每个任务保持专家级、公平且可客观评分。
技能/经验
• 3 年以上亲身从事机器学习推理或服务工作的经验(而非仅学习、销售或管理该领域),或拥有博士学位并在该细分领域有活跃研究
• 在过去 3 年内从事过此类工作,并能列举所使用的系统、工具、数据和规则
• 曾在真实规模(多 GPU 或生产环境服务)下训练、部署或评估模型,而非仅使用笔记本或 API 调用
• 熟悉 PyTorch 或 JAX 内部机制,以及 vLLM、SGLang 或 TensorRT-LLM 等服务技术栈
• 能够解释机器学习结果在无报错情况下出错的原因(数值漂移、指标错误)以及如何发现这些问题
• 熟练掌握 Python 和命令行,以及文件、CSV/JSON、API 或数据库操作
• 能在 2026 年 11 月初开始工作,并能完成每个约 60 至 75 分钟的任务
加分项
• 有 AI 实验室或大语言模型(LLM)基础设施经验
• 曾为 PyTorch、vLLM 或 SGLang 做出贡献
• 有 GPU 性能分析经验
• 熟练使用 Docker 和 Git
• 有编写自动化测试的经验
流程
• 以任务考核代替面试:完成一项考核任务,以便客户评估你的任务设计能力
• 按任务付费;每个任务约需 60 至 75 分钟
• 工作按任务逐个交付,提交内容会获得审阅者反馈
• 预计于 2026 年 11 月初开始
申请时请提及单词 **ASTONISHED** 并附上标签 RMjQwZTo0NTc6ZmYxMjozOWQ6NmQ0Nzo4MGE2Ojc1NzM6NjIzMw==,以表明你已完整阅读本职位发布信息(#RMjQwZTo0NTc6ZmYxMjozOWQ6NmQ0Nzo4MGE2Ojc1NzM6NjIzMw==)。这是一项用于过滤垃圾申请者的测试功能。公司可以通过搜索这些关键词找到完整阅读此信息并证明自己是真人的申请者。
🇬🇧 查看英文原文(Original description)
Overview We are building a benchmark that tests how well AI agents handle real expert work in a terminal. We are hiring ML inference and serving experts to turn real, paid tasks from their jobs into expert-level Terminal-Bench challenges. Each task includes the environment, a working solution, and an automated check that proves the result is correct. ***THIS IS A TASK BASED COMPENSATION MODEL. PLEASE CHECK THE APPLICATION QUESTIONS FOR MORE DETAILS.*** Responsibilities ⢠Design expert-level Terminal-Bench tasks that turn real, paid ML inference work from your own experience into challenges an AI agent must solve end to end. ⢠Define the files, systems and environments each task runs on, such as model serving with streaming, so the challenge is realistic, reproducible and command-line solvable. ⢠Write a working reference solution for every task and verify it runs end to end, sized for about 60 to 75 minutes of expert effort. ⢠Build automated checks that objectively prove a result is correct, including cases where ML outputs go wrong silently, such as numerical drift or metric bugs. ⢠Package tasks with Docker and Git so they are reproducible, and document the systems, tools, data and rules involved so reviewers can validate difficulty. ⢠Iterate on tasks based on reviewer feedback and agent test runs, adjusting difficulty and fixing ambiguities so each task stays expert-level, fair and objectively gradable. Skills/Experience ⢠3+ years doing ML inference or serving work hands-on (not only studying, selling or managing it), or a PhD plus active research in the subfield ⢠Did this work in the last 3 years and can name the systems, tools, data and rules used ⢠Trained, served or evaluated models at real scale (multi-GPU or production serving), not only notebooks or API calls ⢠Hands-on with PyTorch or JAX internals and serving stacks such as vLLM, SGLang or TensorRT-LLM ⢠Can explain how ML results go wrong without errors (numerical drift, metric bugs) and how to catch them ⢠Technical fluency in Python and the command line, and with files, CSV/JSON, APIs or databases ⢠Availability to start by early November 2026 and complete tasks of about 60 to 75 minutes each Nice-to-have ⢠AI lab or LLM infrastructure experience ⢠Contributions to PyTorch, vLLM or SGLang ⢠GPU profiling experience ⢠Docker and Git proficiency ⢠Experience writing automated tests Process ⢠Task vetting instead of an interview: complete a vetting task so the client can assess your task design ⢠Paid per task; each task takes about 60 to 75 minutes ⢠Work is delivered task by task, with reviewer feedback on submissions ⢠Expected start in early November 2026 Please mention the word **ASTONISHED** and tag RMjQwZTo0NTc6ZmYxMjozOWQ6NmQ0Nzo4MGE2Ojc1NzM6NjIzMw== when applying to show you read the job post completely (#RMjQwZTo0NTc6ZmYxMjozOWQ6NmQ0Nzo4MGE2Ojc1NzM6NjIzMw==). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they're human.