TL;DR:: Synavate Labs is almost finished combining Mojo, withj the NLP-based kgcreator, and the generalized gliner-base to create KGLiNER. Mojo is from Modular, kgcreator is an NLP-based tool by Mark Watson, and gliner-base is hosted Hugging Face.
GLiNER is a tool that combines a general "Named Entity Recognition" model and "Knowledge Graph" creation using a contextual approach based on the feedback of an LLM eg. OpenAI GPT-4 or Anthropic's Claude.
The Conceptual Process in Development
- Concatenate multiple pieces of information to form a corpus of knowledge.
- Parse and process the information, ideally with an optimization algorithm that finds the best combination of chunk size, n-dimensional vectorization, etc., for a specific use case. Our product, in the conceptual stage "RAGulator," is based on the concept of automatically tuning the different parameters. In theory, it approximates a loss function, so why not?
- Hope for an appropriate level of determinism, thought we can actually experiment with KG's for more reliable outputs.
- It feels like Reinforcement Learning is relevant here, but I am not so up to speed on it, so we'll cross that bridge when it comes up.
- Backlog: Rust-based knowledge graph with `petgraph` and NetworkX, the Qiskit Quantum Computing tool. There is a lot to learn here, but I do wonder if there are methods in which a QC is going to be superior in graph traversal,
- Therefore, that is perhaps why IBM Qiskit has decided to go that way.
Throughout this project the abilities of the graph data structure to capture relationships has become clear. I is the ability of those entities to be used aspart of a capturing relationships between a "triplet" as primary over nodes in a relational database (lexically hilarious.) This technology is not used and is used extensively in Social Graphs (Netflix, Spotify, Meta etc, X formerly known as Twitter (lulz) etc.
General form of the Graph “triplet” data structure. There can be multiple triplets within a broad graph object
Rather than specifying a fixed schema we are relying on the inherent knowledge of information architecture that LLM's develop. By providing the system with information on the subjects, predicates and objects perhaps we can link them together. Ideally data modelling in complex datasets can become less arduous. Applying the funky new tooling in Machine Learning with features recently available to the community in vector databases and the AI Gateways is of great benefit. For a variety of choice among 100+ inference models, distribution, embedding models and redundancy.
The two toolsets we are borrowing from are "KGCreator":: Authored by Mark Watson, USA:: "gliner_base" @ Hugging Face:: Generalist Model for Named Entity Recognition using Bidirectional Transformer, Authors:: Urchade Zaratiana and Nadi Tomeh and Pierre Holat and Thierry Charnoi (links below)
DALLE-3 on the information highway
KGLiNER is a Knowledge Graph generator based on the Machine Learning technique “Named Entity Recognition.” In the research papers and Mark Watson’s application of the NER model it seems that fixed schemas are the way this kind of learning is done. Our team aims for a broader scope, generating schemas on the fly. For performance, written in Mojo, the latest creation from Chris Latner of the Apple native Swift, and other compiler magicianary (mojiscianry!) We have been drawn toward systems programming for some time and recognize that deep involvement in the space is necessary for getting a grasp… lest we get Rusty (I am a Dad now so those count.)
Mojo by ModularAI is a compiled Python superset, and Machine Learning focused open source (as of this week!) language with high performance in mind. Rumoured to rival C++, or Rust and combined with existing models and exploring the language, KGLiNER enables contextual Named Entity Recognition with custom labels derived from a Large Language Model (LLM) named by Knowledge Graph creation. KGliNER is a (poorly) systems programming refactor of a tool derived from Mark Watson’s foundational kgcreator. Rather than using structured schemas in knowledge graph generation it leverages a bidirectional Transformer for isolation of the named-entities.
Rust is a strongly followed language of almost a decade old at this point versus the 2 years of Mojo. Chris Latner’s vision is a direct compiled superset of Python to maintain it’s ease of use while increase speed through not requiring just-in-time compilation as per dynamic languages like it’s antecedant, Python. It is known for it’s difficulty but the memory safety elements and force strict typing are appealing.
In our attempt to find a general solution to the problem of finding relationships between disparate ideas. we reach for new tools. Data is chunked using the LlamaIndex suite. A matter of interest is the variety of options available when choosing the parameters of an LLM application. How big are your chunks? How about your embedding dimensions? Are you capturing the nuance or the broad strokes of a different type of data? This is the inspiration for the 'RAGulator‘ project we are working on in Rust at the moment.
These projects are both sub-problems within the wide Synapse project is to attempt to see if the application of the inherent language comprehension of any of the LLM Models offered by Portkey is able to effectively be joined in concert with the gliner model to appropriately generate a Knowledge Graph regardless of the types of data that are input.
A system is capable of generalizing the labels that it is looking for. KGLiNER serves as an innovation in the creation and utilization of knowledge graphs. The pretrained generative transformer Gliner_Base being the base modell on Hugging Face.
KGCreator was designed by Mark Watson in Python using his expertise in NLP. Designed to transform textual data into RDF triples with ease, it supports the seamless generation of data for storage in Graph Databases from simple input directories, or directly from a suite of files, with streaming data on the roadmap. This advancement significantly simplifies the construction of knowledge graphs, making it a powerful semantic meaning tool. Engineers and data scientists seeking to organize complex datasets into structured, navigable formats. KGLiNER’s development is inspired by Mark Watson’s work, accessible for further exploration at markwatson.com and his kgcreator on GitHub.
Key features of KGLiNER include:
Automated Conversion: Transforms textual data into RDF triples, facilitating easy data integration into Graph Databases with Named Entity Recognition with labels generated from the text provided.
User-Friendly Interface: The generalized nature of the identification and categorization of categories lends itself to an easy to use platform for ephemeral graph creation.
Portkey AI Gateway: The inclusion of the Portkey AI Gateway enables to focused collection and attribution of labels. We continue to sharpen the tooling but it is a powerful asset with access to 100+ LLM models for inference, retries and a semantic cahce forming a central location for queries.
Open-Source Collaboration: Encourages contributions from the global community, reflecting a commitment to collective advancement and innovation in the field.
Enhanced Data Structuring: Plays a crucial role in the organization of complex datasets, offering a pathway to improved accuracy and contextual understanding for Large Language Models (LLMs).
MLIR Support: We’re not going to pretend we know what this truly is but it sure does sound interesting! Mojo is designed off the back of the heroic efforts of Chris Lattner and his experience in developing compilers not only for Apple’s Swift but the LLVM compiler. The MLIR support as a “next gen compiler” gives low level control beyond our own abilities for this project but ModularAI provides a fantastic interace. We are not completely across the extent of their work due to it’s complexity but enjoy working with Mojo as we expand our knowledge of Systems Programming.
Synavate Labs and others entities are exploring the boundaries of AI/ML applications. By leveraging the latest advancements encapsulated in KGLiNER, in the world of structured data, we’re participating in the slow move forward in the field of Machine Learning and Artificial Intelligence.
As we continue to update our exposure to the world of systems programming we will provide updates on our journey toward “Synapse” – our ultimate technical goal of a join the dots like knowledge experience.
Is it a bird is it a plane? No it’s a supernova Knowledge Graph! DALLE-3
With Love, the Synavate team ::: core@synavate.tech
Aylex Riom 