Another approach (on the Microsoft stack) would be to use a combination of Azure OpenAI services + Azure Cognitive Search to index the structured data. You could then deploy the ChatGPT solution as APIs that could be accessed by your client application.

https://learn.microsoft.com/en-us/shows/ai-show/the-basics-of-prompt-engineering-with-azure-openai-service