Search Results for "240p-test-suite" - Page 4
Sort By:
Showing 182 open source projects for "240p-test-suite"
View related business solutions-
The Most Powerful Software Platform for EHSQ and ESG ManagementChoose from a complete set of software solutions across EHSQ that address all aspects of top performing Environmental, Health and Safety, and Quality management programs.
-
Rezku Point of SaleRezku is an all-inclusive ordering platform and management solution for all types of restaurant and bar concepts. You can now get a fully custom branded downloadable smartphone ordering app for your restaurant exclusively from Rezku.
-
1
Meta Agents Research Environments (ARE)
Meta Agents Research Environments is a comprehensive platform
...Unlike static benchmarks, ARE supports environments where agents must adapt to changes over time and reason over sequences of actions. It interacts with applications and faces uncertainty. The included Gaia2 benchmark offers 800 scenarios across multiple “universes”. It can test reasoning, memory, tool use, and adaptability. Integration with simulated applications/agent APIs (email, file system, etc.). Support for multiple AI model backends/providers. -
2
Aden Hive
Outcome driven agent development framework that evolves
Hive is an open-source agent development framework that helps developers build autonomous, reliable, self-improving AI agents by letting them describe goals in ordinary natural language instead of hand-coding detailed workflows. Rather than manually defining execution graphs, Hive’s coding agent generates the agent graph, connection code, and test cases based on your high-level objectives, enabling outcome-driven agent creation that fits real business processes. Once deployed, agents can capture failure data, evolve automatically to meet their success criteria, and redeploy without constant manual intervention, delivering continual improvement over time. The framework also includes human-in-the-loop nodes, credential management, cost and budget controls, and real-time observability so teams can monitor execution and intervene as needed. ... -
3
openbench
Provider-agnostic, open-source evaluation infrastructure
...With a simple CLI interface (e.g. bench eval <benchmark> --model <model-id>), you can quickly evaluate any model supported by Groq or other providers (OpenAI, Anthropic, HuggingFace, local models, etc.). openbench also supports private/local evaluations: you can integrate your own custom benchmarks or data (e.g. internal test suites, domain-specific tasks) to evaluate models in a privacy-preserving way. -
4
Janus
Unified Multimodal Understanding and Generation Models
Janus is a sophisticated open-source project from DeepSeek AI that aims to unify both visual understanding and image generation in a single model architecture. Rather than having separate systems for “look and describe” and “prompt and generate”, Janus uses an autoregressive transformer framework with a decoupled visual encoder—allowing it to ingest images for comprehension and to produce images from text prompts with shared internal representations. The design tackles long-standing... -
AestheticsPro Medical Spa SoftwareAestheticsPro is the most complete Aesthetics Software on the market today. HIPAA Cloud Compliant with electronic charting, integrated POS, targeted marketing and results driven reporting; AestheticsPro delivers the tools you need to manage your medical spa business. It is our mission To Provide an All-in-One Cutting Edge Software to the Aesthetics Industry.
-
5
Pytorch-toolbelt
PyTorch extensions for fast R&D prototyping and Kaggle farming
A pytorch-toolbelt is a Python library with a set of bells and whistles for PyTorch for fast R&D prototyping and Kaggle farming. Easy model building using flexible encoder-decoder architecture. Modules: CoordConv, SCSE, Hypercolumn, Depthwise separable convolution and more. GPU-friendly test-time augmentation TTA for segmentation and classification. GPU-friendly inference on huge (5000x5000) images. Every-day common routines (fix/restore random seed, filesystem utils, metrics). Losses: BinaryFocalLoss, Focal, ReducedFocal, Lovasz, Jaccard and Dice losses, Wing Loss and more. Extras for Catalyst library (Visualization of batch predictions, additional metrics). ... -
6
Ollama RAG Chatbot
Chat with multiple PDFs locally
...The project is framed as an experiment, but its setup and packaging make it approachable for practical local use as well. It supports running on a local machine or in Kaggle, which lowers the barrier for users who want to test RAG workflows without building everything from scratch. Model support is flexible, with compatibility for both Hugging Face models and Ollama-based models, and the interface is delivered through Gradio for a lightweight user experience. The main value of the project is its ability to process multiple PDF inputs and turn them into a question-answering workflow centered on document retrieval. ... -
7
AIBuildAI
An AI agent that automatically builds AI models
...It provides a structured environment for orchestrating agents that can plan, execute, and refine tasks such as code generation, system design, and iterative improvement loops. The framework is designed to support experimentation with self-improving AI pipelines, allowing developers to test concepts like automated architecture search or adaptive system evolution. It integrates multiple components including prompt management, execution control, and feedback loops to ensure that generated outputs can be evaluated and improved over time. -
8
Kiln
Open source platform for managing, testing, and deploying AI apps
Kiln is an open source platform designed to help developers build, evaluate, and deploy AI-powered applications with greater structure and reliability. It provides a unified environment for managing prompts, datasets, and evaluation workflows, allowing teams to iterate on AI behavior in a controlled and measurable way. Kiln emphasizes reproducibility, enabling users to track changes to prompts and models while comparing outputs across different configurations. Kiln also supports systematic... -
9
FireRedASR
Open-source industrial-grade ASR models
FireRedASR is an industrial-grade family of open-source automatic speech recognition models designed to provide high-precision speech-to-text performance across languages including Mandarin, English, and various Chinese dialects, achieving new state-of-the-art benchmarks on public test sets. The project includes multiple model variants to meet different application needs, such as high-accuracy end-to-end interaction using an encoder-adapter-LLM framework and efficient real-time recognition using attention-based encoder-decoder architectures, giving developers flexibility in balancing performance and resource constraints. ... -
Skillfully - The future of skills based hiringSkillfully transforms hiring through AI-powered skill simulations that show you how candidates actually perform before you hire them. Our platform helps companies cut through AI-generated resumes and rehearsed interviews by validating real capabilities in action. Through dynamic job specific simulations and skill-based assessments, companies like Bloomberg and McKinsey have cut screening time by 50% while dramatically improving hire quality.
-
10
aisuite
Simple, unified interface to multiple Generative AI providers
...Using an interface similar to OpenAI's, aisuite makes it easy to interact with the most popular LLMs and compare the results. It is a thin wrapper around Python client libraries and allows creators to seamlessly swap out and test responses from different LLM providers without changing their code. Today, the library is primarily focused on chat completions. We will expand it to cover more use cases in the near future. Currently supported providers are - OpenAI, Anthropic, Azure, Google, AWS, Groq, Mistral, HuggingFace and Ollama. To maximize stability, aisuite uses either the HTTP endpoint or the SDK for making calls to the provider. -
11
TensorFlow Quantum
Open-source Python framework for hybrid quantum-classical ml learning
...TensorFlow Quantum integrates with the Cirq quantum computing framework to define and manipulate quantum circuits, while leveraging TensorFlow’s infrastructure for optimization, automatic differentiation, and large-scale computation. The library also supports high-performance simulation of quantum circuits, enabling researchers to test and evaluate quantum models even without direct access to quantum hardware. -
12
LongBench
LongBench v2 and LongBench (ACL 25'&24')
...It supports bilingual evaluation in English and Chinese to assess multilingual capabilities across extended contexts. Newer versions of the benchmark introduce extremely long context windows ranging from thousands to millions of tokens, enabling researchers to test the limits of modern long-context models. -
13
AICGSecEval
A.S.E (AICGSecEval) is a repository-level AI-generated code security
...AICGSecEval combines static and dynamic evaluation techniques to analyze generated code for vulnerabilities and functional correctness. The framework includes datasets, test cases, and evaluation metrics that measure how AI programming tools perform across multiple programming languages and vulnerability categories. -
14
LLM Colosseum
Benchmark LLMs by fighting in Street Fighter 3
LLM-Colosseum is an experimental benchmarking framework designed to evaluate the capabilities of large language models through gameplay interactions rather than traditional text-based benchmarks. The system places language models inside the environment of the classic video game Street Fighter III, where they must interpret the game state and decide which actions to perform during combat. This setup creates a dynamic environment that tests reasoning, situational awareness, and decision-making... -
15
Agentless
An agentless approach to automatically solve software development
Agentless is an open-source framework that applies large language models to automatically resolve software development issues without relying on complex autonomous agent systems. The project proposes an alternative approach to AI-driven code repair that avoids the overhead of multi-agent orchestration by using a structured pipeline for identifying and fixing bugs. When solving a problem, the system first performs localization to determine which files, functions, or code segments are most... -
16
MetaCLIP
ICLR2024 Spotlight: curation/training code, metadata, distribution
MetaCLIP is a research codebase that extends the CLIP framework into a meta-learning / continual learning regime, aiming to adapt CLIP-style models to new tasks or domains efficiently. The goal is to preserve CLIP’s strong zero-shot transfer capability while enabling fast adaptation to domain shifts or novel class sets with minimal data and without catastrophic forgetting. The repository provides training logic, adaptation strategies (e.g. prompt tuning, adapter modules), and evaluation... -
17
UpTrain
Your open-source LLM evaluation toolkit
Get scores for factual accuracy, context retrieval quality, guideline adherence, tonality, and many more. You can’t improve what you can’t measure. UpTrain continuously monitors your application's performance on multiple evaluation criterions and alerts you in case of any regressions with automatic root cause analysis. UpTrain enables fast and robust experimentation across multiple prompts, model providers, and custom configurations, by calculating quantitative scores for direct comparison... -
18
promptmap2
A security scanner for custom LLM applications
...Its scanning workflow uses a dual-LLM architecture in which one model acts as the target being tested and another acts as a controller that evaluates whether an attack succeeded. The repository emphasizes broad coverage, including test rules for prompt stealing, jailbreaks, harmful content generation, hate-related outputs, social bias, and distraction attacks. It also supports multiple providers such as OpenAI, Anthropic, Google, xAI, and open-source models through Ollama, making it flexible for both commercial and local deployments. -
19
OpenAGI
When LLM Meets Domain Experts
OpenAGI is a package for AI agent creation designed to connect large language models with domain-specific tools and workflows in the AIOS (AI Operating System) ecosystem. It provides a structured Python framework, pyopenagi, for defining agents as modular units that encapsulate execution logic, configuration, and dependency metadata. Agents are organized in a well-defined folder structure that includes code (agent.py), configuration (config.json), and extra requirements... -
20
Agentex
Open source codebase for Scale Agentex
AgentEX is an open framework from Scale for building, running, and evaluating agentic workflows, with an emphasis on reproducibility and measurable outcomes rather than ad-hoc demos. It treats an “agent” as a composition of a policy (the LLM), tools, memory, and an execution runtime so you can test the whole loop, not just prompting. The repo focuses on structured experiments: standardized tasks, canonical tool interfaces, and logs that make it possible to compare models, prompts, and tool sets fairly. It also includes evaluation harnesses that capture success criteria and partial credit, plus traces you can inspect to understand where reasoning or tool use failed. ... -
21
Petri
An alignment auditing agent capable of exploring alignment hypothesis
Petri is an open-source alignment auditing agent that lets researchers rapidly test concrete safety hypotheses against target models using realistic, multi-turn scenarios. Instead of building bespoke evals, Petri automatically generates audit environments from seed “special instructions,” orchestrates an auditor model to probe a target model, and simulates tool use and rollbacks to surface risky behaviors. Each interaction transcript is then scored by a judge model using a consistent rubric so results are comparable across runs and models. ... -
22
PyTorch Geometric Temporal
Spatiotemporal Signal Processing with Neural Machine Learning Models
The library consists of various dynamic and temporal geometric deep learning, embedding, and Spatio-temporal regression methods from a variety of published research papers. Moreover, it comes with an easy-to-use dataset loader, train-test splitter and temporal snaphot iterator for dynamic and temporal graphs. The framework naturally provides GPU support. It also comes with a number of benchmark datasets from the epidemiological forecasting, sharing economy, energy production and web traffic management domains. Finally, you can also create your own datasets. The package interfaces well with Pytorch Lightning which allows training on CPUs, single and multiple GPUs out-of-the-box. ... -
23
GluonTS
Probabilistic time series modeling in Python
...The dataset consists of a single time-series, containing monthly international passengers between the years 1949 and 1960, a total of 144 values (12 years * 12 months). We split the dataset into train and test parts, by removing the last three years (36 months) from the train data. Thus, we will train a model on just the first nine years of data. Python has the notion of extras – dependencies that can be optionally installed to unlock certain features of a package. We make extensive use of optional dependencies in GluonTS to keep the amount of required dependencies minimal. ... -
24
PySyft
Data science on data without acquiring a copy
Most software libraries let you compute over the information you own and see inside of machines you control. However, this means that you cannot compute on information without first obtaining (at least partial) ownership of that information. It also means that you cannot compute using machines without first obtaining control over those machines. This is very limiting to human collaboration and systematically drives the centralization of data, because you cannot work with a bunch of data... -
25
Recommenders
Best practices on recommendation systems
...Several utilities are provided in reco_utils to support common tasks such as loading datasets in the format expected by different algorithms, evaluating model outputs, and splitting training/test data. Implementations of several state-of-the-art algorithms are included for self-study and customization in your own applications. Please see the setup guide for more details on setting up your machine locally, on a data science virtual machine (DSVM) or on Azure Databricks. Independent or incubating algorithms and utilities are candidates for the contrib folder. ...
