Seonglae ChoUnilever is asking for HAI’s standard best-practice view on what Unilever needs to provide to enable integrated assurance using traces for Procurement GPT (AND for future Agentic Systems)We need a v1 response covering:
1. Trace data objects & schema (v1) – minimum required fields
- prompts in the trace
- tools information in the trace
- timestamp for each action
- subject agent for each action
2. Integration pattern options – event streaming vs batch export, and HAI ingestion expectations
Currently, we support batch export and upload to the platform for agent analysis. We plan to support trace streaming monitoring, though the timeline is undetermined. Possible approaches include webhook-based trace upload to keep the agent graph updated with the current production version, or connecting directly to the source (similar to Azure Foundry Trace) to fetch trace data from our side.
3. Platform / integration prerequisites checklist mapped to Unilever’s bullets below
4. Security & access expectations, including DEV / TEST / PROD readiness
Unilever’s bullets we must respond to (copy pasting as such):
- Data volume, growth outlook, and pull frequency
We expect the agent graph construction and testing frequency to be around once a week. It runs on new traces and generates a single test on the final graph. For each trace, we save a snapshot to track how the state graph is enabled and the data graph for each trace. However, these snapshots are always smaller than the trace content, which means the data volume grows linearly with incoming traces, making it tractable. Testing results size depends on the test cases multiplied by the generated final graph transitions. This is proportional to the sample count for testing but occurs much less frequently since testing is only performed on the final agent graph.
- Upstream/downstream dependencies impacting availability
Upstream dependencies include Unilever's trace generation system and fixed schema, as well as the authentication and authorization system between the Holistic AI platform and Unilever's trace system. If the schema changes or is malformed, traces that do not conform can be ignored. The authentication system between trace generation and the Holistic AI platform is critical for data security. Traces may be missed if the system malfunctions, but the agent state graph is robust to missing trace subsets since other traces cover all states and actions in the agent graph, which does not affect the final output.
For downstream dependencies, we have a main internal infrastructure dependency to save and maintain the analysis. The database and deployed platform's inventory fulfill this role to ensure continued accessibility. Availability impacts typically arise from platform uptime and deployment failures. With the expected weekly agent graph construction from trace streams, the strategy is as follows: if the system goes down → new analysis cannot be performed; if storage fails → analysis results may be lost. However, if traces are preserved on the source side, agent graph results can be reproduced.
- Unique identifiers for end-to-end traceability
We assign a unique identifier to each trace as an artifact key. At the trace level, we record a state traversal history to map trace steps to specific points or windows within the trace content.
- Source-side setup (APIs, feeds, DB access, event triggers)
Currently, we are using export and upload-based methods for trace communication between Unilever and the Holistic AI platform. To enable streaming and automation, there are two implementation approaches: first, API exposure on the Holistic AI platform side that acts as a webhook; second, Unilever opens an API endpoint that can return traces, and Holistic AI can fetch the traces to run analysis. We need to work closely to achieve this and require an additional call to confirm the design direction.
- Performance limits, throttling, and maintenance windows
Each trace-to-agent graph construction requires approximately 30 seconds to 1 minute of analysis time, so this duration is the minimum necessary throttle interval. That means a 10-trace batch run takes around 5 to 10 minutes between calls. Running agent graph analysis on every agent call is highly cost-inefficient and also inefficient from an analysis perspective. For testing, it is sufficient to execute only when the state graph is updated, which will be indicated later through notifications or indicators. For agent graph running, we currently run analysis on all traces, but traces with equivalent diversity do not need to be processed redundantly.