How to Stop Searching for Data and Start Feeding It to AI Agents

Imagine you join a new company as a Senior Data Engineer. You get cloud access, and you see hundreds of tables in BigQuery, thousands of files in Cloud Storage, and a scattering of logs. Where does the current client profile live? Which column in this table is responsible for revenue excluding VAT? Usually, finding answers turns into a detective investigation with endless questions in Slack.

Google decided it was time to do something about this and rolled out Knowledge Catalog. Previously, this service was called Dataplex, but the rebrand here is not just a change of sign. It's an attempt to turn a dry list of tables into a living "knowledge graph" that not only people understand, but LLM agents too.

The knowledge-catalog repository from Google Cloud Platform is a collection of practical tools and examples that show how to get the most out of this platform.

Why Do We Need Another Data Catalog

The problem with most Data Catalogs is that they are static. You describe the data once, and a week later the description is outdated. Knowledge Catalog tries to solve this through AI tooling. It builds a dynamic graph, linking structured tables and unstructured documents with shared business context.

For developers, this means you can now pass an AI agent not just "here's the database API," but "here's a semantic layer where it's clear how the data relates to each other."

The repository contains tools that help automate routine tasks:

• Filling the catalog with metadata without manual input.
• Creating context for RAG systems (Retrieval-Augmented Generation).
• Examples of building agents that "talk" to your data in natural language.

What's Interesting Inside the Repository

The project itself is not a "boxed" product. It's more like a set of building blocks for those building complex data infrastructure on GCP.

I divided the contents into three logical parts.

Tools for Metadata Enrichment

The most boring part of working with data is describing columns. In the repository, there are examples of how to use AI for automatic annotation. The system looks at the table contents, understands that column cust_id contains customer IDs, and automatically writes tags and descriptions. This saves weeks of time on large projects.

Context for AI Agents

This is perhaps the most relevant part. If you've tried writing a chatbot for internal company data, you know: the bot often makes mistakes because it doesn't understand the business logic. For example, it might confuse "registration date" with "last activity date."

Developers from Google suggest using Knowledge Catalog as a linking mechanism. The repository shows how to extract semantic context and feed it to the LLM. As a result, the agent doesn't just search by keywords, but understands relationships in the knowledge graph.

Integration with Cloud Shell

In the README, there's a button for quick launch:

This is convenient if you want to poke around the examples without setting up a local environment and SDK. The entire repository deploys in Google's cloud environment in a couple of minutes.

How to Use This in Practice

Let's say you have a task: build a bot that answers analysts' questions about a huge array of documentation and tables.

Normally, you'd start by writing a simple RAG. But without a data catalog, your RAG will be "blind." Using the developments from this repository, you can:

1. Index all sources through Knowledge Catalog.
2. Use ready-made scripts for metadata extraction.
3. Connect the agent to the dynamic knowledge graph through the proposed code samples.

As a result, the bot will know not only the text of documents, but also the structure of your tables, their owners, and the trust level for this data.

Technical Nuances

The project is distributed under the Apache 2.0 license. It's important to understand that this is not an official Google product, as honestly stated in the disclaimer. This means support falls on the community, and the API of the examples may change.

The main languages of the examples are HTML and Python (although HTML is often highlighted in GitHub metadata due to the documentation structure). Most solutions are tied to Google Cloud services, so if you work in another cloud or on-premise, the project will be useful to you perhaps only as a conceptual example.

Who Should Check Out the Repository

If you're a Data Engineer who's tired of manually filling in descriptions in Data Catalog, or an AI Developer looking for a way to give your agents more context about company data, this project is for you.

It has almost 7,000 stars, which is a lot for a specialized toolkit. This is a good signal: the solutions inside are working and are actually used in production.

It's best to start with the samples folder. That's where the most understandable implementations of how to turn scattered files into a logically connected system are located. The documentation in the repository itself is sometimes laconic, so be prepared to look into the official Google Cloud documentation for Knowledge Catalog and Dataplex.

The project looks like an excellent starting point for those moving from simple chatbots to serious enterprise AI systems, where data quality and context understanding come first.