For a “Hands on your Workflow” I ended up building a Doc Agent Patter I thought may be reusable. I have 1 Doc Agent (1 attachment field) and I reuse it to process multiple attached documents. In this case, they are on the attachment widget, but same basic idea will work for streams that passed in, etc.
I’d love to get feedback on what is here.
Use Case: Need to be able to extract data from a large pdf. Issue is that the doc is too large for doc agent to handle.
Solution: Split pdf. In this case I’m working with large bpm type diagrams. I split the original pdf into 2 page pieces. Then read the pieces/parts in sequence and extract information from all parts.
For example WF.GA.COB was the original and then I split it into 6 pieces. Then iterated through the parts (1-6) in sequence and passed them to the doc agent to extract the information I needed.
Doc Agent snippet below:
Below diagram in red is the solution that was implemented. Extract/Split PDF directly followed by reading the pdf parts (_partx) in order and passing them to the doc agent for information extraction. This was followed by data cleanup DT to simply move data back into properties that downstream apps expected.
When I wrote this, I had a dual purpose. 1.) I needed this activity for this solution and 2.) I wanted to write something meaningful with the AI Assistant. So this activity was iterated on and completely written with AI Assistant. I have to admit, it documented it better than I would of done on my own…
End result: All information extracted from all _partx and combined in a single table view for the end user to easily manage.
.Just wondering if you know the maximum document size that can be ingested without splitting. At what point should we consider splitting a document into multiple parts to ensure optimal processing?