Skip to content

improvement(mothership): agent model dropdown validations, markers for recommended models#4213

Merged
icecrasher321 merged 8 commits intostagingfrom
improvement/agent-dropdown-model
Apr 18, 2026
Merged

improvement(mothership): agent model dropdown validations, markers for recommended models#4213
icecrasher321 merged 8 commits intostagingfrom
improvement/agent-dropdown-model

Conversation

@icecrasher321
Copy link
Copy Markdown
Collaborator

Summary

  • Mark recommended + speedOptimized models
  • Add deterministic validation error for invalid model ids (correctly handles dynamic models like ollama, etc)

Type of Change

  • Other: Context Improvement

Testing

Tested manually

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 17, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Apr 18, 2026 1:50am

Request Review

@cursor
Copy link
Copy Markdown

cursor Bot commented Apr 17, 2026

PR Summary

Medium Risk
Introduces stricter server-side validation for model inputs on blocks using getModelOptions, which can reject previously-accepted values and affect workflow edit operations. Also changes VFS block schema output for model options, which could impact downstream consumers expecting the prior shape.

Overview
Adds server-side validation for model fields on combobox/input subblocks that use getModelOptions, trimming whitespace, accepting provider-prefixed dynamic ids (e.g. ollama/...), and returning a deterministic "Unknown model id" error with suggested ids for uncataloged values.

Extends provider model definitions with recommended, speedOptimized, and deprecated flags, and updates VFS block schema serialization to emit these markers alongside static model options plus a dynamicProviders note/prefix list; updates tests to cover the new model validation behavior across agent and router_v2 while ensuring non-catalog model fields (e.g. HuggingFace) are unaffected.

Reviewed by Cursor Bugbot for commit de7390b. Configure here.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 17, 2026

Greptile Summary

This PR adds recommended, speedOptimized, and deprecated tier flags to ModelDefinition, marks selected models accordingly, and introduces a deterministic model-ID validation step in the copilot's validateValueForSubBlockType. The validation rejects hallucinated static-provider IDs (e.g. claude-sonnet-4.6) while allowing dynamic-provider prefixes (ollama/, fireworks/, etc.) and unrecognized bare IDs (local Ollama tags). The VFS serializer is extended to propagate tier flags — including inheritance for reseller providers — and a dynamicProviders note is added to block schemas to help the AI handle non-catalog IDs correctly.

Confidence Score: 4/5

Safe to merge; the two findings are P2 design observations that don't affect correctness of the primary validation path.

The core validation logic is correct and well-tested. The startsWith(base+"-") permissiveness is an intentional tradeoff for date-suffixed variants and doesn't break the primary hallucination-detection use case. The duplicate dynamic-provider list is a maintenance risk but not a current defect. Scoring 4 rather than 5 because the suffix-matching gap could admit hallucinated IDs that look like -, which marginally weakens the stated goal of catching bad model IDs.

apps/sim/providers/models.ts (isKnownModelId suffix check) and apps/sim/lib/copilot/vfs/serializers.ts (DYNAMIC_PROVIDERS_NOTE sync)

Important Files Changed

Filename Overview
apps/sim/providers/models.ts Adds recommended/speedOptimized/deprecated flags to ModelDefinition and selected models; introduces isKnownModelId, getRecommendedModels, and suggestModelIdsForUnknownModel — the startsWith(base+"-") fallback is intentionally permissive but accepts any suffix after a known ID.
apps/sim/lib/copilot/tools/server/workflow/edit-workflow/validation.ts Wires model-ID validation into validateValueForSubBlockType using a function-reference equality check against getModelOptions; logic is sound but returns the trimmed value for the catalog path regardless of whether the original was trimmed.
apps/sim/lib/copilot/vfs/serializers.ts Enriches VFS block schema with tier flags (recommended/speedOptimized/deprecated) and inherits them for reseller providers via RESELLER_BASE_PREFIX; DYNAMIC_PROVIDERS_NOTE prefixes list is manually maintained separately from the canonical DYNAMIC_MODEL_PROVIDERS in models.ts.
apps/sim/lib/copilot/tools/server/workflow/edit-workflow/validation.test.ts Good test coverage added for the new model validation path, including catalog matches, hallucinated IDs, empty values, dynamic prefixes, date-suffixed variants, and whitespace trimming; follows project vi.hoisted + vi.mock pattern correctly.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[validateValueForSubBlockType\nfield='model', type='combobox'] --> B{subBlockConfig.options\n=== getModelOptions?}
    B -- No --> C[Return value as-is\nnon-catalog path]
    B -- Yes --> D{trimmed === ''}
    D -- Yes --> E[Return empty string\nvalid: true]
    D -- No --> F[isKnownModelId\ntrimmed]
    F --> G{Exact catalog match\nor startsWith base+'-'?}
    G -- Yes --> H[valid: true\nreturn trimmed]
    G -- No --> I{Dynamic provider\nprefix? ollama/ vllm/ etc.}
    I -- Yes --> H
    I -- No --> J{Matches static\nprovider modelPattern?}
    J -- Yes --> K[valid: false\nUnknown model id error\n+ suggestions]
    J -- No --> L[valid: true\nunrecognized bare id\ne.g. local Ollama tag]
Loading

Reviews (1): Last reviewed commit: "mark a few more models:" | Re-trigger Greptile

Comment thread apps/sim/providers/models.ts Outdated
Comment thread apps/sim/lib/copilot/vfs/serializers.ts
Comment thread apps/sim/providers/models.ts Outdated
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/providers/models.ts
Comment thread apps/sim/lib/copilot/vfs/serializers.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/providers/models.ts
Comment thread apps/sim/lib/copilot/vfs/serializers.ts Outdated
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/lib/copilot/tools/server/workflow/edit-workflow/validation.ts Outdated
Comment thread apps/sim/providers/models.ts
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/lib/copilot/tools/server/workflow/edit-workflow/validation.ts Outdated
@icecrasher321
Copy link
Copy Markdown
Collaborator Author

bugbot run

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit de7390b. Configure here.

@icecrasher321 icecrasher321 merged commit 319e0db into staging Apr 18, 2026
14 checks passed
@icecrasher321 icecrasher321 deleted the improvement/agent-dropdown-model branch April 18, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant