Multimodal Search
AI-powered search across your data
Ocular’s AI-powered search engine enables natural language search across your entire data catalog, making it easy to find specific content within images, videos, and other multimodal data.
Save search results as indexes, which can then be leveraged across multiple projects.
Overview
Traditional search systems rely on metadata like filenames or manual tags, making it difficult to search through unstructured visual data. Ocular’s multimodal search solves this by combining:
- Semantic understanding of visual content
- Natural language processing
- High-dimensional vector embeddings
- Hybrid retrieval system
This allows you to search using natural language queries that describe what you’re looking for, rather than exact matches of metadata.
The search interface on Foundry:
Key Features
Natural Language Queries
Search using descriptive language:
- “Person entering doorway at night”
- “Forklift near loading dock”
- “Red car in parking lot”
Cross-Modal Search
Find relevant content across different types of media:
- Images
- Videos
- Documents
- Audio (transcribed content - coming soon!)
Semantic Similarity
Results are ranked by semantic relevance, not just keyword matching:
- Understands context and meaning
- Finds visually similar content
- Groups related items together
Timestamp-Level Results
For video content, search returns specific timestamps where relevant content appears.
How it Works
How it Works
1. Data Indexing
When data is ingested into the Catalog:
- Content is analyzed by AI models
- Visual features are extracted
- High-dimensional embeddings are generated
- Metadata is processed and indexed
2. Search Processing
When you perform a search:
- Your query is converted to a semantic embedding
- The system performs hybrid retrieval:
- Vector similarity matching
- Metadata filtering
- Relevance ranking
- Results are returned ranked by relevance
3. Results Display
Search results show:
- Preview thumbnails
- Relevance scores
- Timestamps (for video)
- Associated metadata
- Quick actions for further processing
Using Search Effectively
Best Practices
-
Be Descriptive
- Use complete phrases instead of keywords
- Include relevant details about what you’re looking for
- Specify context when needed
-
Refine Results
- Use filters to narrow down results
- Sort by different criteria
- Save frequent searches
-
Leverage for Workflows
- Create indexes from search results
- Build datasets for training
- Identify patterns and anomalies
Example Queries
Good queries are descriptive and specific:
Avoid single keywords or vague terms:
Search capabilities are continuously improving as we add new models and features. Check the changelog for updates.