The AI homepage (Coming soon!) has a section for UMD Chatbots.
There are two types of UMD AI Chatbots: Departmental and Academic.
This can be created for a department or a smaller unit within a department. It can be internal or public-facing.
Academic chatbots are for specific university academic courses. Only a controlled list of users, such as students, professors, and TAs, can access this bot. Extra features like quiz mode, Panopto transcript/video indexing, and disabling the bot during tests are available.
You can request a chatbot using the upcoming Chatbot Provisioning Request; upon successful completion, you will receive an email with information about your bot and the next steps. The form will be available soon.
Yes, there is a monthly fee depending on the type of chatbot. The fee is described on the Chatbot Service Now page.
You need a UMD email, be assigned as a Chatbot Admin, and use CAS authentication to access the Chatbot Admin UI web page.
We can be reached by email at ese-sws@umd.edu.
The UMD AI Chatbot Admin Application is a web page where you can find the link to your Google Drive and add web pages to scrape and control other settings for your bot.
To ensure UMD Academic AI Chatbots provide a consistent experience for all students, two tasks must be completed before students can access the bot. First, as the bot admin, you need to answer a set of common topics in the Scripted Questions section of the AI Chatbot UI. Next, you must provide sufficient documents and/or web pages for your bot. Finally, questions will be generated and tested against the bot. These three steps help ensure a consistent and helpful experience for students.
Your UMD AI Chatbot has a Google Drive where you can copy many file types, including doc, Google Doc, pdf, xls, Google Sheet, PPT, Google Slides, text, and text transcripts. You can also specify website URLs to have their HTML data scraped to answer questions. Finally, you can enter extra data directly into the Scripted Questions section of the AI Chatbot UI.
Yes, however, only PDFs are supported at this time. Also, since websites are recursively scraped, you need to review the list of PDFs on the Ingested Data Chatbot Admin UI web page in the Web PDFs tab and toggle the desired entries. This will trigger the web scrape process and will be completed along with the next file scan cycle instead of waiting until the next day's regularly scheduled web scrape.
The AI Chatbot UI User Config page provides a link to a stand-alone question window. This can be used to test your bot and share the link with potential users. A popup chatbot window can also be embedded in a web page if you have a web page administrator who can integrate it. The bot can not be integrated directly into Canvas.
This type of AI Chatbot answers questions quickly. If it takes more than a few seconds, there may be network interference or another issue. Contact the AI Chatbot team if the bot is not responding quickly enough.
Yes, sources are quoted in line with the answer using clickable links that open the sources box below the answer. There are also links to the material used to answer the question.
Yes, data placed in the public folder in Google Drive will provide source links to the documents. Data placed in the private folder will indicate a source but not give a link to the source material.
Yes, clicking on the thumbs-up will mark the question so it shows up in the Thumbs Up tab on the AI Chatbot UI Question Review page. Clicking thumbs down will first ask for a brief description of the problem, then mark the question so it shows up in the Thumbs Down tab.
When you log into the AI Chatbot Admin UI, the first page you see is Question Review. This has all the questions asked in the current month. Questions can be marked as reviewed, request developer review with comments you enter, or inspected. Finally, the data the bot used to answer the question can be inspected here.
Yes. The bot is configured to consider the last N questions when working on the current question. This, however, depends on what N is set to and whether there is enough space to hold all the current and previous data when building the complete set of information going to the LLM to generate an answer. Contact the AI Chatbot team to discuss the history, performance, or settings involved here.
The AI Chatbot Admin UI Question Review page has an icon shaped like a piece of paper. It pops up a window that gives access to all of the source text chunks, chunks from previous questions, the prompt, and previous questions and answers. While the LLM can craft its answer drawing from any of that information, the one or two it says it used are highlighted.
As the bot admin, you can set the bot's greeting and disclaimer in the AI Chatbot UI "User Config" page. If you want to change your bot's name, please contact the AI Chatbot team.
Yes, contact the UMD AI Chatbot office with a request to enable quiz mode. This feature will allow bot users to ask for a multiple-choice quiz generated from the data you provided to the bot. The AI Chatbot team may need to make other updates to your bot's configuration to enable the best quiz experience.
Mostly. If you create a directory in your Google Drive called Assessments and put all the test material under it, the bot will offer resources about user questions that closely match assessment questions instead of responding with the actual answer. As the bot admin, you can set the text in the response when this feature is triggered.
Yes, As the bot admin, you can use the AI Chatbot UI to toggle the bot's Test Mode on the User Config page. You can also set the text in the response when this Quiet Mode feature is used. Don't forget to turn the bot back on.
Yes, the bot will mask things like social security numbers and other PII at the source, so the data is not stored in the system.
Yes, the LLM used to generate the answer to the question is configured to detect inappropriate and dangerous questions and provide a declined or supporting response.
Yes. The User Analytics page in the AI Chatbot UI has a tab that graphs questions asked vs. time. There are also tabs for thumbs up and down questions asked.
Yes. The User Analytics page in the AI Chatbot UI has an Intent Occurrence tab that graphs the number of times each chunk is used to answer a question. Each chunk has an intent name, just a few words describing what the chunk is about or where it came from. This can help you understand the data that users are looking for the most so it can be refined as needed.
The bot administrator should review the questions, looking for incorrect answers and thumbs-down responses. The AI Chatbot Admin UI Question Review page has a way to mark questions as reviewed to make it easier to keep track of progress.
When you find questions answered wrong, it's frequently because your bot ingested missing, conflicting, bad, or too much data.
Data is missing when the bot's response indicates it does not have information in its context to answer the question. Add files, web pages, or scripted questions to provide more data the bot can choose from.
The bot may be forced to choose from conflicting data and pick the wrong source. In this case, use the AI Chatbot Admin UI Question Review page to click the sources icon for the conflicting question. Find the source data chunk(s) that are not preferred and remove or fix them from the file or web page. If it's an unwanted web page, add it to the URLs to not Scrape list in the AI Chatbot Admin UI User Config page.
This type of AI Chatbot works best when clear semantic matches exist to the questions. If too many chunks of data match, the one you want may not match. In this case, you can reduce redundant data or increase the number of chunks returned to answer questions. Contact the AI Chatbot team if you want to adjust settings like this.
Currently, Google Drive is scanned for changes about every 20 minutes, and website data is scraped once a day, usually early in the morning.
The Chatbot Admin UI has an "Ingested Data" web page that shows the data ingested into your bot as multiple chunks for each file. The files are chunked to make it easier for the bot to match questions to the most relevant data it knows from your files.
The bot can only work with the data it is provided. It keeps a list of broken web pages in the Chatbot Admin UI Ingested Data page, which you can use to find out what is out of date. Also, in the Chatbot Admin UI Ingested Data page, you can add URLs that are not included during the web scrape process. These could be broken URLs or just old/archived data you don't want the bot to reference.
Yes, if you are making website changes and want them ingested right away and you have PDFs on your website, toggle on or off a small PDF. This will trigger the web scrape process for your bot, which will be completed along with the next Google Drive cycle.
You can do that. Contact the UMD AI Chatbot office and include your Chatbot name and the ID of the Google Drive from which you would like your Chatbot to read. The ID is the long string of characters and numbers in the last part of the URL when looking at the top level of Google Drive.
You can tell the bot to skip files in Google Drive by renaming them with BOT_IGNORE in the front of the filename. This behaves like deleting the file from the drive, and the bot will remove it from its database. Note: Make sure there are no leading spaces in front of BOT_IGNORE, or it will not work.
The UMD AI Chatbot uses a custom Retrieval Augmented Generation (RAG) technique that combines similarity-based data retrieval with generative-based artificial intelligence (AI) models.
No. The UMD AI Chatbot system is designed to answer questions only based on the data you provide. While it uses a powerful AI model to formulate answers from that data, it is instructed not to make up answers if it can not confidently create an answer based on the chunks of your ingested data it found related to the question. This direction can be modified. Contact the AI Chatbot team if you want to change your bot's behavior prompt.
The bot looks for chunks of your data that closely match the question. These chunks work best when they are concise and hold information about one topic. Chunks come from paragraphs of document files, rows of spreadsheets, short periods of transcribed audio, portions of PDFs, sections of web pages, HTML, and scripted questions.
Documents other than PDFs are usually broken into chunks by looking for paragraphs. Paragraphs separated by newlines help the bot break up the document into better chunks. This can be verified by looking at the chunks in the Chatbot Admin UI Ingested Data page. The text of the chunk is shown in the right column. Only the first 200 characters are displayed in the table cell, so it's best to verify chunks by looking at their beginning information.
PDFs are chunked by looking for text breaks and font size changes. Making these gaps larger may help the bot find natural breaks in information. Multiple columns on a page are ok. If a topic starts at the bottom of one page and continues to the top of another page, it will be broken into different chunks. Tables and grids of information are hard for the bot to turn into textual chunks of data. If some critical data is in a table, it may need to be entered into the bot separately as Scripted Questions in the Chatbot Admin UI.
Keep each entry focused on one topic. In the Question cell, enter multiple versions of the question separated by newlines. Keep the total number of words in the answer under about 1000. Any links in the text should have https:// at the beginning.
Your bot can be configured with various AI models and other settings. It is best to contact the AI Chatbot team with your concerns. They can adjust the model, number of chunks used, amount of data passed to the model, and other settings to help the bot find and use the best data.
This AI Chatbot only uses semantic search to find the chunks that best match the words in the question. If too many chunks are needed to answer the question or the question has multiple parts, the bot cannot find and use all the information necessary to answer the question.
Possibly, Reach out to Axel Persaud in the UMD Division of IT. The AI Chatbot team has experience creating Chatbots with AI Agents that access more complex information and databases. They can create a solution to access your unique data.