Table of Contents
Introduction
Welcome to the user manual for the Generative AI Virtual Agent Admin Console. This guide will assist you in managing and configuring your Virtual Agent effectively, leveraging its features, and troubleshooting common issues. Whether you are a new admin or an experienced one, this manual will provide you with valuable insights to optimize your Virtual Agent's performance.
Getting started
Accessing the Admin Console
To access the admin console, follow these steps:
- Visit the Admin Console's URL.
- Log in with your UMD credentials.
- Access the dashboard.
Dashboard overview
The dashboard provides an overview of the Virtual Agent's performance and key metrics. You can navigate through various sections using the menu on the left side.
Virtual Agent configuration
This section allows you to configure settings specific to your Virtual Agent.
- Select the Virtual Agent you want to configure in the top left dropdown.
- Navigate to the Virtual Agent Config section in the left hand sidebar.
The sections of the Virtual Agent Configuration are as follows:
- A description of the type of Virtual Agent selected (Academic or Departmental).
- A link to your Virtual Agent.
- Connectors to the sources of information to be ingested into the Virtual Agent’s data. The different kinds of connector currently include: Google Drive, Web Scraper and Service now. By default most Virtual Agents start with just a Google Drive connector. To add others, email our support team.
- WebScraper
- No websites are scraped unless a URL is within the webscraper collection table.
- Click ‘Add’ and insert the url as the title. The scrape process for this website starts as soon as the new entry is created and saved. Click the trash icon in the actions column to delete a url.
- Refer to the Ingested Data page to see the count of web chunks go up. This process takes at least a minute and can run longer if the website being scraped has many child pages.
- If an entire starting URL website in this widget is no longer needed, it can be deleted. This will remove all the chunks that were ingested for this website and its children.
- If a ‘WebScraper’ section is not visible on your page and you would like to add web content to your Virtual Agent, please email our support team.
- WebScraper
-
- Google Drive
- A link to the Google Drive folder within which files can be uploaded to your Virtual Agent.
- Files added to the Google Drive are automatically ingested into the Virtual Agent’s data. When files are updated or removed, the data associated with them is also updated or removed from the Virtual Agents data.
- Within the Google Drive folder there will be a Private and Public folder by default.
- When used to generate an answer, files within the Public folder will appear in the Virtual Agent's answer sources as clickable links (Only users that have access to the Google Drive will be able to enter the Drive with the link).
- When used to generate an answer, files within the Private folder will appear in the Virtual Agent's answer sources as "Private Source"
- If you have a Panopto video you would like to ingest, please email our support team.
- If you would like to change the Google Drive folder your Virtual Agent is using, please email our support team.
- ServiceNow
- A UMD KB Article base URL will have to be provided in order to use this functionality. The user will then have to provide a Knowledge base category and all articles and PDF attachments under that knowledge base will be ingested.
- If a ‘ServiceNow’ section is not visible on your page and you would like to add ServiceNow content to your Virtual Agent, please email our support team.
- Google Drive
- Customization of the agent name, greeting message, and follow-up messages.
- Some sections of this page may be visible but non-configurable because they can affect how the Virtual Agent functions. This can be seen by the lock icon in the field. If you’d like to edit any of these sections please email our support team.
Ingested Sources
This section allows you to view and configure the data ingested by your Virtual Agent. This is also the place to set PDFs found in web pages to be included in the Virtual Agent’s ingested sources.
- Select the Virtual Agent you want to configure in the top left dropdown
- Navigate to the Ingested Sources section in the left hand sidebar.
The top panel shows you an overview of the size and type of the information ingested by your Virtual Agent. Each chunk represents about a paragraph of text, so each source will make multiple chunks of data.
View Ingested Sources
Click the Ingested Sources tab to view what data is ingested in your Virtual Agent.
Each chunk of data has the following fields:
- Ingested Date
- Type (The type of source the data was ingested from, the possible types are as follows):
- wdoc (A Word Document file).
- gdoc (A Google Document file).
- ppt (A PowerPoint file).
- sheet (A Google Sheet file).
- excel (An Excel file).
- csv (A csv file).
- txt (A txt file, typically a video transcript).
- pdf (A pdf file).
- web (A webpage).
- qna (A Scripted Question and Answer).
- Intent (A descriptive name for the chunk of data).
- Title (The name of the file or webpage ingested to make that chunk of data).
- Image Url (If this paragraph is a description of an image, the link to that image is here.)
- Text (The text that the chunk of data represents)
.
View and toggle Broken webpages from Ingested webpages
- Click the Broken Webpages tab to view what webpages are flagged as broken but are set to be ingested into your Virtual Agent.
- Click on the toggle to include or not exclude the corresponding webpage.
View the Ingested Details tab
- The Ingested Details tab shows ingested sources broken up into embeddings as they appear in the database. A text preview is provided in this section that displays only the first 100 characters of the text.
Data Analysis
Question review
This section allows you to review questions asked by users and manage their status. You can filter questions by year, month (for Virtual Agents with higher use this may be by day), and review status.
- Navigate to the Question Review section.
- On the left you can select the year and month to view the corresponding questions.
- Use the tabs to filter questions by their review status (All, Unknown, Known, Thumbs Up, Thumbs Down, Reviewed, Needs Dev Eval, Dev Feedback, Test).
- Use the search bar to filter your results.
- The questions have 6 fields:
- Actions (Actions available to you to review the questions, see below).
- Timestamp (The date and time the question was asked).
- Intent Used (The label associated with the source used to answer the question).
- Question (The question asked by the user).
- Response (The response generated by the Virtual Agent).
- Session ID (The generated session id that is refreshed when a user refreshes the chat page).
- There are 4 action buttons that allow you to review the questions:
- Mark as Reviewed (Resolve the question and sort it into the Reviewed tab).
- Mark as Test (Resolve the question and sort it into the Test tab).
- Request Developer Review (Submit the question with reasoning to receive developer attention and feedback on that question, sorts the question into the Needs Dev Eval tab, once a developer reviews said question it will be sorted into the Dev Feedback tab).
- See Query Sources (View sources and the prompt that the Virtual Agent used to create the answer to that question).
Data sources
Scripted questions
This section is used to manage scripted questions that the Virtual Agent can use to provide predefined answers.
Add scripted questions
- Navigate to the Scripted Questions section.
- Click on the Plus sign(+) button to add a new scripted question.
- Fill in the intent, script question, and answer.
Edit scripted questions
- Navigate to the Scripted Questions section.
- Select the intent, script question, or answer of the scripted question you want to edit.
- Change the value to your desired entry.
Remove Scripted questions
- Navigate to the Scripted Questions section.
- Click on the Dash icon button next to a scripted question to delete it.
- Confirm the deletion.
Analytics
This section provides insights into user interactions with the Virtual Agent. You can view metrics such as the total number of questions asked, unknown questions, thumbs up, and thumbs down.
View analytics
- Navigate to the Virtual Agent Analytics section.
- Select the month to view the metrics.
- Analyze the graphs and data provided to understand user interactions.
Troubleshoot
Admin Console not accessible
- Check your internet connection.
- Ensure the admin console URL is correct.
- Contact support if the issue persists.
Virtual Agent not responding correctly
- Review the response setup and training data.
- Ensure integrations are correctly configured.
Support
If you encounter issues that you cannot resolve, email our support team.