AI workflows for AI and machine learning.
Building and running AI and machine-learning features and models.
Who does this work
- Machine learning engineer
- AI engineer
- Applied scientist
Updated
Examples of the workflows identified
- Build the evaluation set for an AI featureIllustrative
- Check a production model for driftIllustrative
- Compare candidate models on the evaluation setIllustrative
- Review an AI feature's failures from production logsIllustrative
The workflows identified
Marked illustrative: an example of the work, identified, with no workflow template yet.
Build the evaluation set for an AI featureAssembles representative and difficult test cases for an AI feature from real inputs, each with the expected result and how it will be scored.Illustrative · not built yet
Check a production model for driftCompares recent inputs and outputs of a model in production with the data it was built on, showing shifts in inputs, scores and accuracy where results are known.Illustrative · not built yet
Compare candidate models on the evaluation setRuns candidate models or prompt versions against the evaluation set and compares quality, cost and speed, with the cases where they disagree.Illustrative · not built yet
Review an AI feature's failures from production logsSamples low-rated, corrected or abandoned AI outputs from the logs and groups them into failure types with examples and counts, for the team to prioritize fixes.Illustrative · not built yet
Workflow templates
There is no workflow template for AI and machine learning yet: request one below and we build it with you.
Request this workflow
Tell us the AI and machine learning work you need and the team that does it, and we build the workflow with you from your own procedures.
Where to go next
All of software engineeringEvery area of the function, with workflow templates and examples.Map your team in the PlannerPut this work in your plan, with the roles that do it.
Other areas of software engineering