README.md

---
library_name: sentence-transformers
pipeline_tag: sentence-similarity
license: apache-2.0
tags:
- embeddings
- semantic-search
- sentence-transformers
- presentation-templates
- information-retrieval
base_model: sentence-transformers/all-MiniLM-L6-v2
datasets:
- cyberagent/crello
language:
- en
---

# Field-adaptive-bi-encoder

## Model Details

### Model Description
A fine-tuned SentenceTransformers bi-encoder model for semantic similarity and information retrieval. This model is specifically trained for finding relevant presentation templates based on user queries, descriptions, and metadata (industries, categories, tags) as part of the Field-Adaptive Dense Retrieval framework for structured documents.

**Developed by:** Mudasir Syed (mudasir13cs)

**Model type:** SentenceTransformer (Bi-encoder)

**Language(s) (NLP):** English

**License:** Apache 2.0

**Finetuned from model:** sentence-transformers/all-MiniLM-L6-v2

**Paper:** [Field-Adaptive Dense Retrieval of Structured Documents](https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544)

### Model Sources
- **Repository:** https://github.com/mudasir13cs/hybrid-search
- **Paper:** https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544
- **Base Model:** https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2

## Uses

### Direct Use
This model is designed for semantic search and information retrieval tasks, specifically for finding relevant presentation templates based on natural language queries. It implements field-adaptive dense retrieval for structured documents.

### Downstream Use
- Presentation template recommendation systems
- Content discovery platforms
- Semantic search engines
- Information retrieval systems
- Field-adaptive dense retrieval applications
- Structured document search and ranking

### Out-of-Scope Use
- Text generation
- Question answering
- Machine translation
- Any task not related to semantic similarity or document retrieval

## Bias, Risks, and Limitations
- The model is trained on presentation template data and may not generalize well to other domains
- Performance may vary based on the quality and diversity of training data
- The model inherits biases present in the base model and training data
- Model outputs are optimized for presentation template domain

## How to Get Started with the Model
```python
from sentence_transformers import SentenceTransformer
import torch

# Load the model
model = SentenceTransformer("mudasir13cs/Field-adaptive-bi-encoder")

# Encode text for similarity search
queries = ["business presentation template", "marketing slides for startups"]
embeddings = model.encode(queries)

# Compute similarity
from sentence_transformers import util
cosine_scores = util.cos_sim(embeddings[0], embeddings[1])
print(f"Similarity: {cosine_scores.item():.4f}")

# For retrieval tasks
documents = [
    "Professional business strategy presentation template",
    "Modern marketing presentation for tech startups",
    "Financial report template for quarterly reviews"
]

# Encode queries and documents
query_embeddings = model.encode(queries)
doc_embeddings = model.encode(documents)

# Find most similar documents
similarities = util.cos_sim(query_embeddings, doc_embeddings)
print(f"Top matches: {similarities}")
```

## Training Details

### Training Data
- **Dataset:** Presentation template dataset with descriptions and queries
- **Size:** Custom dataset of presentation templates with metadata
- **Source:** Curated presentation template collection from structured documents
- **Domain:** Presentation templates with field-adaptive metadata

### Training Procedure
- **Architecture:** SentenceTransformer (all-MiniLM-L6-v2) with contrastive learning
- **Base Model:** sentence-transformers/all-MiniLM-L6-v2
- **Loss Function:** Triplet loss with hard negative mining / Multiple Negatives Ranking Loss
- **Optimizer:** AdamW
- **Learning Rate:** 2e-5
- **Batch Size:** 16
- **Epochs:** 3

### Training Hyperparameters
- **Training regime:** Supervised learning with contrastive loss
- **Hardware:** GPU (NVIDIA)
- **Training time:** ~2 hours
- **Max Sequence Length:** 512 tokens

## Evaluation

### Testing Data, Factors & Metrics
- **Testing Data:** Validation split from presentation template dataset
- **Factors:** Query-description similarity, template relevance, field-adaptive retrieval performance
- **Metrics:** 
  - MAP@K (Mean Average Precision at K)
  - MRR@K (Mean Reciprocal Rank at K)
  - NDCG@K (Normalized Discounted Cumulative Gain at K)
  - Cosine similarity scores
  - Recall@K

### Results
- **MAP@10:** ~0.85
- **MRR@10:** ~0.90
- **NDCG@10:** ~0.88
- **Performance:** Optimized for presentation template retrieval in structured document search
- **Domain:** High performance on field-adaptive dense retrieval tasks

## Environmental Impact
- **Hardware Type:** NVIDIA GPU
- **Hours used:** ~2 hours
- **Cloud Provider:** Local/Cloud
- **Carbon Emitted:** Minimal (efficient fine-tuning)

## Technical Specifications

### Model Architecture and Objective
- **Base Architecture:** Transformer-based bi-encoder (all-MiniLM-L6-v2)
- **Objective:** Learn semantic representations for field-adaptive dense retrieval
- **Input:** Text sequences (queries, descriptions, and metadata)
- **Output:** 384-dimensional dense embeddings
- **Pooling:** Mean pooling strategy

### Compute Infrastructure
- **Hardware:** NVIDIA GPU
- **Software:** PyTorch, SentenceTransformers, Transformers

## Citation

**Paper:**
```bibtex
@article{field_adaptive_dense_retrieval,
  title={Field-Adaptive Dense Retrieval of Structured Documents},
  author={Mudasir Syed},
  journal={DBPIA},
  year={2024},
  url={https://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE12352544}
}
```

**Model:**
```bibtex
@misc{field_adaptive_bi_encoder,
  title={Field-adaptive Bi-encoder for Presentation Template Search},
  author={Mudasir Syed},
  year={2024},
  howpublished={Hugging Face},
  url={https://huggingface.co/mudasir13cs/Field-adaptive-bi-encoder}
}
```

**APA:**
Syed, M. (2024). Field-adaptive Bi-encoder for Presentation Template Search. Hugging Face. https://huggingface.co/mudasir13cs/Field-adaptive-bi-encoder

## Model Card Authors
Mudasir Syed (mudasir13cs)

## Model Card Contact
- **GitHub:** https://github.com/mudasir13cs
- **Hugging Face:** https://huggingface.co/mudasir13cs
- **LinkedIn:** https://pk.linkedin.com/in/mudasir-sayed

## Framework versions
- SentenceTransformers: 2.2.2+
- Transformers: 4.35.0+
- PyTorch: 2.0.0+