Exploring the World of Small Language Models Fine-Tuned on Synthetic, Domain-Specific Data (This is a WIP)
# Starting Details
initial_model = microsoft/Phi-3-small-8k-instruct
dataset = gretelai/synthetic_text_to_sql (filtered for relevant domains to Cyber security and Defence)
final_model = synavate/cyber-tuned-phi3-8k
MLOps = Weights & Biases (Experiment Tuning and Tracing)
Fine-tuning optimisation: Unsloth.ai
Large language models are renowned for their extensive knowledge base, having been trained on over a trillion tokens (sub-words). Microsoft Research has recently released some small language models with impressive results. My goal is to evaluate the effectiveness of using a suite of these models as agents in Docker containers, and to determine if they can function satisfactorily. This blog post will be somewhat technical as I document my experiments.
I’ll also be using Daniel So’s “Unsloth” fine-tuning library from Build Club Australia. Shout out to the Build Club.
Questions Explored in This Exercise:
Question 1: Degree of Performance Degradation
- Methods/Analysis: Comparing pre- and post-fine-tuning performance metrics.
- Conclusions: Identifying any significant performance degradation and potential causes.
Question 2: Model Collaboration in an Agentic Environment
- Methods/Analysis: Testing the model’s ability to interact with other agents in a simulated environment.
- Conclusions: Evaluating the success of model collaboration and identifying any limitations.
Question 3: Influence of GPT-4
- Methods/Analysis: Introducing GPT-4’s influence into the agentic environment and observing changes in performance.
- Conclusions: Assessing whether GPT-4’s presence mitigates collaboration issues and enhances overall performance.
Lastly, I will deploy this model in a Google Cloud Platform environment as part of a security big data cloud analytics project. This involves using Redpanda (Fast Kafka) for streaming data/message queue services and BigQuery. I am curious to see if there is too much divergence in the query language. I will cross that bridge when I get there. As I mentioned, this is a WIP fun exercise. Given that Redpanda will be a cornerstone of Orion, it will be a valuable experience to become familiar with it.
🤗Fine-Tuning Techniques
Distillation: Trains a smaller model to mimic the behavior of a larger pre-trained model, optimizing efficiency while maintaining performance.r.
PEFT (Parameter-Efficient Fine-Tuning): Fine-tunes pre-trained models by adjusting only a small subset of parameters, reducing computational costs.
LoRA (Low-Rank Adaptation): Enhances transformer models by injecting and training low-rank matrices within each layer, minimizing the number of trainable parameters.
QLoRA (Quantized Low-Rank Adaptation): Combines low-rank adaptation with weight quantization to achieve efficient fine-tuning with reduced memory and computational requirements.
Full Fine-Tuning: Updates all parameters of the pre-trained model, offering high flexibility at the cost of increased computational resources.
Appendix A – Colab Gist (As of 180724)
https://gist.github.com/orionhunts-ai/60d0c5895ae312cd21a828c8294dc57d.jsThis journey will continue for some time as I continue to work on the core of the Orion AI Threat Intelligence platform. There is infinite opportunity in this space. Join us!
Peace and love, world.
Snyata
Aylex Riom 