Power Mode Editor plugin for Neovim; uselessly awesome, annoyingly entertaining (for a few minutes). Supports screen shake, firewall (a la cacafire), and rich configuration across everything (because why not)
Excited to show you a new project I have been working on: the K8s Agent Orchestration Framework (KAOS) which helps you deploy and manage distributed multi-agent systems at scale If you want to support, do try it out, add an issue or give it a star!
The KAOS Framework addresses some of the pains of taking multi-agent / multi-tool / multi-model systems to hundreds or thousands of services. It started as an experiment to build agentic copilots, and has progressed as a fun endevour building distributed systems for A2A, MCP Servers, and model inference!
The initial release comes with a few key features including:
1) Golang control plane to manage Agentic CRDs;
2) Python data plane that implements a2a, memory, tool / model mgmt;
3) React UI for CRUD+debugging, and;
4) CI/CD setup with KIND/pytest/ginko/etc.
Sharing some interesting (preliminary) results for the 2024 State of Production ML Survey:
Deploying ML: 36% take 1-3 months, 21% 3-6 months
Experiment Tracking: 42% use MLFlow, 10% Spreadsheets
Feature Stores: 53% use none, 28% Custom-built
Vector DB: 55& use none, rest unconsolidated
Training: 27& use custom-built, 21% Databricks
Serving: 56% use custom-built (+ FastAPI/Flask)
Monitoring: 50% use none, 24% Custom-built
Diversity: Only 4% identify as female
This is a community initiative and the data will provide all of us in the community with actionable insights to improve the ecosystem. We aim for this input will help create a comprehensive overview of common practices, tooling preferences, and challenges faced when deploying models to production, ultimately benefiting the entire ML community
We are opening this survey until the end of October, and we'll publish the results for the community to derive useful insights! If you can please take two minutes to share your experience: https://bit.ly/state-of-ml-2024
You can also check out the preliminary results here: https://ethical.institute/state-of-ml-2024 - we are building an interface for basic slice and dice to enable extracting further insights (but still early WIP so feedback appreciated). Final results / report will be published end of October!
For some background, this project started after seeing various renowned machine learning frameworks like Pytorch and Tensorflow integrating Vulkan as a backend. The Vulkan SDK offers a great low level interface that enables for highly specialized optimizations - however it comes at a cost of highly verbose code which requires 800-2000 lines of code to even begin writing application code. This has resulted in each of these projects having to implement the same baseline to abstract the non-compute related features of the Vulkan SDK.
This large amount of non-standardised boiler-plate can result in limited knowledge transfer, higher chance of unique framework implementation bugs being introduced, etc. We are aiming to address this with Kompute. As of today, we are now part of the Linux Foundation, and slowly contributing to the cross-vendor GPGPU revolution.
Some of the key features / highlights of Kompute:
* C++ SDK with Flexible Python Package
* BYOV: Bring-your-own-Vulkan design to play nice with existing Vulkan applications
* Asynchronous & parallel processing support through GPU family queues
* Explicit relationships for GPU and host memory ownership and memory management: https://kompute.cc/overview/memory-management.html
* Robust codebase with 90% unit test code coverage: https://kompute.cc/codecov/
* Mobile enabled via Android NDK across several architectures
