Table of Contents
Introduction
Welcome to the user manual for the Generative AI Virtual Agent Admin Console. This guide will assist you in managing and configuring your Virtual Agent effectively, leveraging its features, and troubleshooting common issues. Whether you are a new admin or an experienced one, this manual will provide you with valuable insights to optimize your Virtual Agent's performance.
Getting started
Accessing the Admin Console
To access the admin console, follow these steps:
- Visit the Admin Console's URL.
- Log in with your UMD credentials.
- Access the dashboard.
Dashboard overview
The dashboard provides an overview of the Virtual Agent's performance and key metrics. You can navigate through various sections using the menu on the left side.
Virtual Agent configuration
This section allows you to configure settings specific to your Virtual Agent.
- Select the Virtual Agent you want to configure in the top left dropdown.
- Navigate to the Virtual Agent Config section in the left hand sidebar.
The sections of the Virtual Agent Configuration are as follows:
- A description of the type of Virtual Agent selected (Academic or Departmental).
- A link to your Virtual Agent.
- A link to the Google Drive folder within which files can be uploaded to your Virtual Agent.
- Within the Google Drive folder there will be a Private and Public folder by default.
- When used to generate an answer, files within the Public folder will appear in the Virtual Agent's answer sources as clickable links (Only users that have access to the Google Drive will be able to enter the Drive with the link).
- When used to generate an answer, files within the Private folder will appear in the Virtual Agent's answer sources as "Private Source"
- If you have a Panopto video you would like to ingest, please email our support team.
- If you would like to change the Google Drive folder your Virtual Agent is using, please email our support team.
- Access Control (Add Directory IDs to limit the access to the Virtual Agent to only the users present, an Empty Access Control gives access to all users with the link).
- Customization of the agent name, greeting message, and follow-up messages.
- Connectors including URLs you want the Virtual Agent to scrape and ingest and URLs you do not want the Virtual Agent to scrape and ingest, by default:
- No websites are scraped unless a URL is within URLs to Scrape.
- By default, URLs linked within the webpages in URLs to Scrape will be scraped as well unless they are within URLS to NOT Scrape.
- No PDFs will be ingested from webpages.
Ingested data
This section allows you to view and configure the data ingested by your Virtual Agent.
- Select the Virtual Agent you want to configure in the top left dropdown
- Navigate to the Ingested Data section in the left hand sidebar.
The top panel shows you an overview of the size and type of the information ingested by your Virtual Agent. Each chunk represents about a paragraph of text, so each source will make multiple chunks of data.
View Ingested Data
Click the Ingested Data tab to view what data is ingested in your Virtual Agent.
Each chunk of data has the following fields:
- UUID (A unique identifier for the data).
- Type (The type of source the data was ingested from, the possible types are as follows):
- wdoc (A Word Document file).
- gdoc (A Google Document file).
- ppt (A PowerPoint file).
- sheet (A Google Sheet file).
- excel (An Excel file).
- csv (A csv file).
- txt (A txt file, typically a video transcript).
- pdf (A pdf file).
- web (A webpage).
- qna (A Scripted Question and Answer).
- Intent (A descriptive name for the chunk of data).
- Title (The name of the file or webpage ingested to make that chunk of data).
- Text (The text that the chunk of data represents).
View and toggle Ingested webpages
- Click the Ingested Webpages tab to view what webpages are ingested into your Virtual Agent.
- Click on the toggle to include or not exclude the corresponding webpage.
View and toggle Ingested webpages
- Click the Broken Webpages tab to view what webpages are flagged as broken but are set to be ingested into your Virtual Agent.
- Click on the toggle to include or not exclude the corresponding webpage.
View and toggle Ingested webpages
- Click the Web PDFs tab to view what web pdfs were found to be in webpages ingested into your Virtual Agent.
- Click on the toggle to include or not exclude the corresponding pdf.
Data Analysis
Question review
This section allows you to review questions asked by users and manage their status. You can filter questions by year, month, and review status.
- Navigate to the Question Review section.
- On the left you can select the year and month to view the corresponding questions.
- Use the tabs to filter questions by their review status (All, Unknown, Known, Thumbs Up, Thumbs Down, Reviewed, Needs Dev Eval, Dev Feedback, Test).
- Use the search bar to filter your results.
- The questions have 6 fields:
- Actions (Actions available to you to review the questions, see below).
- Timestamp (The date and time the question was asked).
- Intent Used (The label associated with the source used to answer the question).
- Question (The question asked by the user).
- Response (The response generated by the Virtual Agent).
- Session ID (The generated session id that is refreshed when a user refreshes the chat page).
- There are 4 action buttons that allow you to review the questions:
- Mark as Reviewed (Resolve the question and sort it into the Reviewed tab).
- Mark as Test (Resolve the question and sort it into the Test tab).
- Request Developer Review (Submit the question with reasoning to receive developer attention and feedback on that question, sorts the question into the Needs Dev Eval tab, once a developer reviews said question it will be sorted into the Dev Feedback tab).
- See Query Sources (View sources and the prompt that the Virtual Agent used to create the answer to that question).
Data sources
Scripted questions
This section is used to manage scripted questions that the Virtual Agent can use to provide predefined answers.
Add scripted questions
- Navigate to the Scripted Questions section.
- Click on the Plus sign(+) button to add a new scripted question.
- Fill in the intent, script question, and answer.
Edit scripted questions
- Navigate to the Scripted Questions section.
- Select the intent, script question, or answer of the scripted question you want to edit.
- Change the value to your desired entry.
Remove Scripted questions
- Navigate to the Scripted Questions section.
- Click on the Dash icon button next to a scripted question to delete it.
- Confirm the deletion.
Analytics
This section provides insights into user interactions with the Virtual Agent. You can view metrics such as the total number of questions asked, unknown questions, thumbs up, and thumbs down.
View analytics
- Navigate to the Virtual Agent Analytics section.
- Select the month to view the metrics.
- Analyze the graphs and data provided to understand user interactions.
Troubleshoot
Admin Console not accessible
- Check your internet connection.
- Ensure the admin console URL is correct.
- Contact support if the issue persists.
Virtual Agent not responding correctly
- Review the response setup and training data.
- Ensure integrations are correctly configured.
Support
If you encounter issues that you cannot resolve, email our support team.