---
library_name: peft
base_model: michiyasunaga/BioLinkBERT-large
tags:
- medical
- cardiology
- embeddings
- domain-adaptation
- lora
- sentence-transformers
- sentence-similarity
language:
- en
license: apache-2.0
---

# CardioEmbed-BioLinkBERT

**Domain-specialized cardiology text embeddings using LoRA-adapted BioLinkBERT-large**

This is the **best performing model** from our comparative study of 10 embedding architectures for clinical cardiology.

## Performance

| Metric | Score |
|--------|-------|
| Separation Score | **0.510** |
| Similar Pair Avg | 0.811 |
| Different Pair Avg | 0.301 |
| Throughput | 143.5 emb/sec |
| Memory | 1.51 GB |

## Usage

```python
from transformers import AutoModel, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModel.from_pretrained("michiyasunaga/BioLinkBERT-large")
tokenizer = AutoTokenizer.from_pretrained("michiyasunaga/BioLinkBERT-large")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "richardyoung/CardioEmbed-BioLinkBERT")

# Generate embeddings
text = "Atrial fibrillation with rapid ventricular response"
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
outputs = model(**inputs)
embeddings = outputs.last_hidden_state.mean(dim=1)
```

## Training

- **Training Data**: 106,535 cardiology text pairs from medical textbooks
- **Method**: LoRA fine-tuning (r=16, alpha=32)
- **Loss**: Multiple Negatives Ranking Loss (InfoNCE)

## Citation

```bibtex
@article{young2024comparative,
  title={Comparative Analysis of LoRA-Adapted Embedding Models for Clinical Cardiology Text Representation},
  author={Young, Richard J and Matthews, Alice M},
  journal={arXiv preprint},
  year={2024}
}
```

## Related Models

This is part of the CardioEmbed model family. See [richardyoung/CardioEmbed](https://huggingface.co/richardyoung/CardioEmbed) for more models.