Skip to content

Working Group: Omni Models

Overview

Lead: This working group is led by Jeremy Fowers, whose handle is @jeremyfowers on GitHub and @jfowers_amd on Discord.

Background: Lemonade Omni Models combine multiple models and backends together to present omni-modal capabilities (e.g., chat and image editing in the same model) that would otherwise not be readily available in local AI.

Why: Omni-modality enables natural interactions between end-users and local AI systems, and we believe it is essential for mass adoption of local AI.

Goal: Improve Omni Models to become a staple of local AI apps and use. The condition for this is not yet apparent, so we will start by iterating on some obvious next steps.

Contributing

Please see the general contribution guidelines, then contact @jfowers_amd on Discord before getting started to discuss the roadmap.

Roadmap

Roadmap items may be high-level objectives that may span multiple issues and PRs. Specific issues and PRs can be found in the Omni Models GitHub Milestone.

  • [ ] Develop application-specific LMX skills, system prompts, and tools to be more useful for workflows such as image generation.
  • [ ] Update the Lemonade website and top-level README to emphasize the developer journey, towards embedding lemond and integrating LMX models.
  • [ ] Lemonade Mix (LMX) omni models are available and useful in Open WebUI. Publish a blog explaining LMX and showing use cases.
  • [ ] Publish Halo Tales, a reference application based on LMX models. Update Lemonade to smooth over any rough edges. Then, publish a blog explaining the LMX developer journey and learnings from making Halo Tales.
  • [ ] Migrate LMX model definitions to Hugging Face to make them searchable as models. Make sure Lemonade is shown under Use This Model on the model card.
  • [ ] To be continued, based on the learnings from the above...