Loading...
Microsoft Co-Pilot ImplementationCopyrighted November 4, 2024 City of Dubuque WORK SESSION # City Council ITEM TITLE: 5:30 PM - Microsoft Co -Pilot Implementation SUMMARY: Chief Information Officer Chris Kohmann will discuss the City of Dubuque's strategy and plans for the implementation of Microsoft CoPilot. SUGGUESTED DISPOSITION: ATTACHMENTS: 1. MVM Memo 2. Staff Memo to City Manager Microsoft CoPilot Implementation Strategy 3. Presentation -Uploaded 11.4.24 Page 11 of 2498 THE C DUjIBQTE Masterpiece on the Mississippi TO: The Honorable Mayor and City Council Members FROM: Michael C. Van Milligen, City Manager SUBJECT: Microsoft Co -Pilot Implementation DATE: October 31, 2024 Dubuque AIFAWca Ciq ni I 2007-2012.2013 2017*2019 Chief Information Officer Chris Kohlmann is submitting information for the Work Session on Microsoft Co -Pilot Implementation held on November 4, 2024, at 5:30 p.m. k�4 S4-Z�22'� Mic ael C. Van Milligen MCVM:sv Attachment cc: Crenna Brumwell, City Attorney Cori Burbach, Assistant City Manager Chris Kohlmann, Chief Information Officer Page 12 of 2498 THE CITY OF DUB TE Masterpiece on the Mississippi TO: Michael C. Van Milligen, City Manager FROM: Chris Kohlmann, Information Services Manager SUBJECT: Microsoft Co -Pilot Implementation DATE: October 31, 2024 INTRODUCTION Dubuque III-lmerin Co 2007-2012.2013 2017*2019 The purpose of this memo will be to introduce and provide background information as content for a work session of the City Council to discuss the City of Dubuque's strategy and plans for the implementation of Microsoft CoPilot. The presentation will introduce some proposed direction on governance and best practices in the city on application and use of Artificial Intelligence (AI) also referred more specifically as "Generative Al." BACKGROUND Artificial Intelligence or Al has been around since the early 50's and is known for the concept of "machine learning" using patterns of data to anticipate predicted outcomes. In the city of Dubuque, Al using these concepts is currently deployed in the city for technologies such as: Cybersecurity - recognizing patterns of malicious data traffic, such as ransomware and stopping it. STREETS - patterning vehicular traffic and predicting future needs for transportation. Smarter Cities pilots and programs — applying large amounts of data to assist residents in making better, more informed reasons about water, electricity, discards, transportation, and health. Generative Al is a set of technologies that leverages large (very large) volumes of data along with machine learning (ML) techniques to produce content based on inputs from the users known as prompts. Generative Artificial Intelligence (AI) can generate text, images, code, or other types of content based on the patterns they've "learned" using Large Language Learning models (LLM.) These tools are evolving rapidly and are still the subject of active research: improving our understanding of how they work, and the impacts of their use in society. These tools are not actual intelligence in the human sense, rather, they are very sophisticated models that predict what the language, text, Page 13 of 2498 or video that satisfies the prompt should be. (Source: City of Boston Interim Guidelines for Using Generative Al Version 1.1 Prepared by Santiago Garces, Chief Information Officer, City of Boston Published: 5/18/2023 Applies to: all City agencies and departments.) As we continue to explore innovative solutions to advance our operations and services, through a lens of being an equitable, data -driven high-performance government, the integration of Microsoft Copilot presents a significant opportunity to enhance our capabilities. Microsoft 365 Copilot for Government Community Cloud (GCC) leverages large language models (LLMs) with an organization's data to enhance productivity for government entities. This AI -powered assistant is designed to automate routine tasks, provide data -driven insights, and enhance collaboration, allowing staff to focus on what matters most —serving residents effectively. Through integration with the Microsoft applications already used by staff, Copilot offers real-time intelligent assistance to boost creativity, productivity, and skills. It's important to note that LLMs have been trained on an extensive body of information, some of which has been retained, giving them a broad understanding of language, the world, reasoning, and text manipulation. However, we should use them as engines rather than stores of knowledge. DISCUSSION What is Copilot? Microsoft 365 Copilot is a generative artificial intelligence (AI) support element for Microsoft 365 productivity applications (such as Excel, the "new Outlook," PowerPoint and Word.) A version made for US governments has not yet been made available for licensing. Per Microsoft, the Government Community Cloud (GCC) enterprise licensing is planned with two rollouts. The first of two scheduled launch phases for Copilot for Microsoft 365 GCC is provisionally set to happen sometime in Q4 of 2024. The deadline is still subject to approvals from the US government. This phase will include: • Microsoft App Copilots for Word, Excel, PowerPoint, • Outlook (Mail and Calendar) streamlining email and calendar management • Teams Chat/Channel Copilot enhancing collaboration across departments via messaging • Intelligent Meeting Recap generating a comprehensive summary of your meetings, including key points, action items, and decisions mad The second phase planned for some time in Q1 of 2025 will introduce Copilot in additional services to further streamline workflow and services including Teams Meetings, OneNote, Stream, Planner, OneDrive and Loop. Per Microsoft, these Page 14 of 2498 additional tools will deepen Al integration within your organization, enabling more advanced task management, content creation, and collaboration capabilities as you continue your journey toward digital transformation. Ecosystem of CoPilot Microsoft 365 Apps �T�, EXM3"0114 Microsoft Grapt Microsoft 365 Copilot Large Language 00 Model o00 When a user enters a query into Copilot, the following things happen: 1. The user's prompt is grounded on Microsoft Graph data, and a modified prompt is created (pre-processing) 2. The modified prompt is then sent to the LLM for a response or app command (processing) 3. The response is then grounded again against the Microsoft Graph (post - processing) 4. Finally, the response is sent back to the user. Why is "Grounded" in bold? The issue is that LLMs have a "hallucination" problem, they come up with an answer even if they don't know the answer. For certain queries, this is probably fine — but for some contexts such as government ordinance or budgets it can be quite harmful. Because LLMs are designed to be predictive, they "predict" responses. Grounding is designed to prevent this risk. The primary motivation for grounding is that LLMs are not databases, even if they possess a wealth of knowledge. Microsoft has implemented Retrieval Augmented Generation (RAG) as the primary technique for grounding. RAG is a process for retrieving information relevant to a task, Page 15 of 2498 providing it to the language model along with a prompt, and relying on the model to use this specific information when responding. Benefits and Applications of Microsoft Co -Pilot Microsoft 365 Copilot GCC is designed to enhance productivity and streamline workflows across various public sector use cases. Here are some potential use cases to consider- 1 . Efficiency Improvements: Reduction in manual tasks and streamlined workflows. 2. Policy Analysis: Quickly synthesize vast amounts of data to inform decision - making. Organizations could use Copilot to analyze trends and draft evidence - based policy recommendations, allowing them to respond more effectively to emerging issues. 3. Citizen Services: Improve response times and accuracy in addressing citizen inquiries. Copilot can help draft personalized responses to common questions about government services, permits, or regulations, ensuring that citizens receive timely and accurate information. 4. Enhanced Decision -Making: Copilot in Excel can be used to perform complex scenario analyses and generate easy -to -understand reports and executive summaries for stakeholders. 5. Project Management and Increased Collaboration: Copilot can help create project timelines, assign tasks, and generate progress reports, keeping multi - agency efforts on track and ensuring that projects are completed on time and within budget. 6. Training and Knowledge Sharing: Facilitate continuous learning within your organization. Use Copilot to create training materials, FAQs, and knowledge bases, ensuring your staff stays up to date with the latest procedures and regulations, and fostering a culture of ongoing professional development. Governance and Policv Considerations I've had an opportunity to do some high-level research/discussion/education around Generative Al using resources from the National League of Cities (NLC,) T-Mobile, Google, and Microsoft along with review of other local governments as we anticipate implementation of Generative Al. The inclination is to look first at the application of available technologies; however, this is a technology that will require the governance of generative Al to ensure its ethical and responsible deployment. Toward this effort, The National League of Cities through NLC RESOLUTION 2024-42 Adopted at the 2023 City Summit has outlined Local Principles for The Governance of Generative Artificial Intelligence. Page 16 of 2498 In addition, for Al governance to be effective it is vital to include: Accountability and Transparency • Accountability in Al entails establishing mechanisms to assign clear roles and responsibilities for the system's actions, outcomes, and overall organizational impact. • Transparency is the clear and open disclosure of how Al systems operate and make decisions. Stakeholders should be informed about the Al system's capabilities, limitations, and the data it uses. Equity and Inclusiveness • Equity and Inclusion must be considered in all aspects of tools such as Al when used to engage residents, analyze data, assess risk, and provide data driven decisions. Inclusiveness in Al signifies the commitment to designing systems that understand, respect, and serve the diverse needs of a global user base without bias. Privacy and Security • Security encompasses safeguarding the entire Al system, including the model, infrastructure, and associated data, from unauthorized access and malicious intents. • Privacy relates to protecting user data, ensuring interactions remain confidential, and that the Al does not unintentionally divulge sensitive or personal information. Robustness and Safety • Robustness refers to the ability of an Al system to continue to function even when faced with unexpected or adverse conditions. • Safety refers to the ability of an Al system to operate without causing harm to humans, property, or the environment. Explainability and Interpretability • Explainability refers to the ability to describe in human terms why a model made a particular decision. • Interpretability refers to the ability to understand the inner workings of a model. An interpretable model is one where we can understand the process it uses to arrive at a decision. Applying these principles on governance would result in Best Practices of Generative Al including: Page 17 of 2498 1) The use of Al should support the work of our employees to deliver better, safer, more efficient, and equitable services and products to our residents. There is value to be had in the use of technology, particularly new generative Al, but there are also risks, some of which will not be apparent or fully understood upfront. 2) Embracing a culture of responsible experimentation, where we maintain control and understanding of the use of new Al tools while we develop new uses that drive efficiency, civic engagement, high performance data driven governance or other outcomes in service of our residents. 3) Fact checking and reviewing all generative Al outputs is vital to the use of Al. Humans are ultimately responsible for whatever products or outcomes they publish, regardless of whether Al is used. This can be a valuable tool for local leaders. However, like human -made decisions, Al systems have the potential for bias and discrimination depending on their training, the data they use, and their ultimate application. 4) Artificial intelligence is not a magical solution. Al utilizes statistical processes designed to find patterns in data. Extra care should be exercised when Al is used to inform consequential decisions or to provide risk assessments that impact people's lives. The best known of these new tools are developed for commercial purposes. While they can be adapted for mission -driven work by public professionals, it is important to maintain service to the public at the center of our work. 5) Disclosure when generative Al has been used. Residents expect transparency from their local government. Regardless of the tools, use of technologies such as generative Al are a reflection of the City and ourselves. We are stewards of the public trust, and we will use tools respectfully and responsibly. 6) Equity and inclusion must be considered in all aspects of tools such as Al when used to engage residents, analyze data, assess risk, and provide data driven decisions. Our work should uplift these communities and connect them more effectively with the resources they need to thrive. 7) Every technology tool that we use has an impact on the security of our overall environment, and the privacy and digital rights of our residents. The core data stewardship principal is: "Do not share sensitive or private information." The information input into generative Al prompts is not inherently private and therefore could be vulnerable to security threats or could be taken by the vendor providing the model to further train their technology. Page 18 of 2498 ACTION STEP Please review and let me know if you have any additional questions. The presentation will cover the background outlined in this memo along with the next steps proposed for implementation of CoPilot in the City of Dubuque. Cc: Crenna Brumwell, City Attorney Sources used for this memo: Microsoft's Knowledge Graph and Copilot - by Arda Capital Microsoft Copilot for Microsoft 365 GCC GA Update: Empowering Public Sector Innovation Mastering Generative Al Governance: Best Practices I AISecHub Page 19 of 2498 Dubuque City Council Work Session November 4, 2024 Page 20 of 2498 How did we� get here? j copilot J Benefits and Application Principles —i Policy Licensing Milestones Page 21 of 2498 S(Z(G' A.I. TIMELIN 950 TURING TEST Computer scientist Alan Turing proposes a test for machine intelligence. If a machine can trick humans into thinking it is human, then it has i0telligence 1999 1955 A.I. BORN Term 'artificial intelligence' is coined by computer scientist, John McCarthy to describe "the science and engineering of making intelligent machines' 2002 1961 UNIMATE First industrial robot, Unimate, goes to work at GM replacing humans on the assembly line 2011 AIBO I ROOMBA SIRI Sony launches first First mass produced Apple integrates Siri, consumer robot pet dog autonomous robotic an intelligent virtual Ai80 (Al robot) with vacuum cleaner from assistant with a voice skills and personality iRobot learns to navigate interface, into the that develop overtime and clean homes iPhone 4S 1964 ELIZA Pioneering chatbot developed by Joseph Weizenbaum at MIT holds conversations with humans 2011 WATSON IBM's question answering computer Watson wins first place on popular $1 M prize television quiz show Jeopardy 1966 SHAKEY The `first electronic person' from Stanfoi Shakey is a general- purpose mobile rob that reasons about its own actions 2014 EUGENE Eugene Goostman, a chatbot passes the Turing Test with a third of judges believing Eugene is human A.1 ■ WINTER Many false starts and dead -ends leave A.I. in the cold 2014 Amazon launches Alexa, an intelligent virtual assistant with a voice interface that completes shopping tasks 1997 DEEP BLUE Deep Blue, a chess - playing computer from IBM defeats world chess champion Garry Kasparov 2016 TAY Microsoft's chatbot Tay goes rogue on social media making inflammatory and offensive racist comments r : • KISMET Cynthia Breazeal at MIT introduces KISmet, an emotionally intelligent robot insofar as it detects and responds to people's feelings ©• AlphaG❑ 2017 ALPH- ;O GoogIe's A.I. AlphaGo beats world champion Ke Jie in the complex board game of Go, notable for its vast number (2170)of possible positions g _ Timeline of Recent Generative Al Events https://ediscoverytoday.com/2023/10/17/ a-timeline-of-recent-generative-ai-events- artificial-intelligence-trends/ • b Junes Nov Feb mp"� 2019 2020 2022 023 GPT-1 CPT-2 GPT-3 ChatGPT ChatGPT release release release Released Pius (117M (1.55 (175B Subscription parameters) parameters) parameters) Service Released • kL. Mar 20ok9 2023 2-023 2023 2023 2023 GPT-4 Anthropic GPT-4 Microsoft openAl suffers Released launched passes bar announces data breach (2,76T Claude exam (90th Copilot (patched/ parameters) percentile) (available announced 11/1) 3/241 1111ar 31, May 15, May 23, IN 2023 2023 2023 Google ChatGPT Italy OpenAl Microsoft launched plugin banned launches announces Bard support ChatGPT ChatGPT Bing use of made (restored iOS app ChatGPT available 4/28) June 2, June 25, July 7, July 13, Sept 25, 2023 023 2023 2023 2023 Libel case ChatGPT Silverman/ FTC opens ChatGPT image filed against officiates others file investigation and voice ChatGPT in Colorado copyright into OpenAl capabilities Georgia wedding suit against announced OpenAl Page 23 of 2498 2024 Trends Mainstream applications will continue to embed Al AI -powered Virtual Assistants Will Be Integrated Further Into Our Lives. ILI The limits of historical large language models will be pushed Generative Al Tools Will Have To Solve Problems — Or Fade Away. Customer Sentiment Data Will Become More Powerful. Large language models shift to "Multimodal Models" incorporating text, images, 3-D, audio, video, music, physical action and more. re, Al will redefine creative industries �a Rising Al demand mandates governance and `humans in the loop: cif 1..Alit- Tz"r.=-1I A MfNHT"Me. - . I U&MIM.b�ib�y�cf� 1 � �1�1� 1• •11 •1 "1 •11" ". •" .1 � .� •1 / � .1� �1 1. 11- � •1. 1 Page 24 of 2498 Microsoft Copilot for Microsoft 365 architecture --------- Plug -ins Bing , -----------------I /-----------------y i Dataverse + Power Platform Services ' A i ----------------------------- Pre-processing « , i Grounding VW , , , i Compliance and , Purview --------------------- , , Microsoft Graph , , I I I I I -------------------------------------------- Prompts, responses, and grounding data i aren't used to train foundation models , , prompt Large Language ANEW , Modified Model Azure Operg instance is maintained by , q,rFrpt Microsoft.OpenAl has no access to thedata * orthe modelAlk , , LLM i , sponse RAI , RAI is performed on inputprompt and output results , r-----------------------------------------------------------------�* f I* Data flow ( = all requests are encrypted via HTTPS and wss;(A 1 1 Your context and content (emails, files, meetings, chats, calendars, and contacts) Customer , Microsoft, ----------------------------------/ User prompts are sent to Copilot Copilot accesses Graph + (optional) Web + Other services for grounding Copilot sends modified prompt to Large Language Model (LLM) Copilot receives LLM response Copilot accesses Graph for Compliance and Purview • Large Language Models - LLMs Have a "Hallucination" Problem • LLM's come up with an answer even if they don't know the answer. • Because LLMs are designed to be predictive, they "predict" responses. • Grounding is designed to prevent this risk. • The primary motivation for grounding is that LLMs are not databases, even if they possess a wealth of knowledge. • Microsoft has implemented Retrieval Augmented Generation (RAG) as the primary technique for grounding. RAG is a process for retrieving information relevant to a task, providing it to the language model along with a prompt, and relying on the model to use this specific information when responding. Page 26 of 2498 50 More than Dace per day 5 Daily 5-6 days per week 2 40 �-4 days per week 6 30 1-2 days per week 1 Less than Dare per week 22 10 Never 54 Page 27 of 2498 Adobe Firefly 1 450 Anthrop is Claude 3 50 40 ball- E 30 Microsoft osoft Copilot 12 iS cogiI e G em irri (Forme rly Ba rd) 9 20 Oth er (PI ease exp la i n bel ) 12 10 None of the above 51 � m I in Page 28 of 2498 "I always verify the accuracy of content generated by Al tools" More Details Strongly Disagree 4 Somewhat Disagree 2 Neither Agree Nor Disagree 2 19 Sornew hat Agree 10 Strong Iy Ag ree 35 Page 29 of 2498 "Generative Ais a tool. We are responsible for the outcomes of our tools. For example, if autocorrect unintentionally changes sword -changing the meaning of something we wrote, we are still responsible for the text. Technology enables our work, it does not excuse our judgment nor our accountability." Santiago Garces., CIO, Boston Page 30 of 2498 PoLicyPrincipLes of Al What does this mean? • Supporting our staff to deliver better, safer, more efficient and equitable services and products to our residents • We are stewards of the public, and we will use Al tools respectfully and responsibly. EMPOWERMENT INCLUSION AND TRANSPARENCYAND . When we act transparently, we build trust, and we RESPECT ACCOUNTABILITY gain the ability to learn collectively and be accountable for actions • There is value to be had in the use of technology, O particularly new generative Al, but there are also risks • Understanding that there is an impact on the security of our overall environment, and the INNOVATION AND PRIVACYAND PUBLIC PURPOSE privacy and digital rights of our residents are most RISK MANAGEMENT SECURITY important. • It is important to maintain service to the public at the center of our work Adapted from the City of Boston Guidelines for Using Generative Al Page 31 of 2498 Licensing - Why Not Yet? Office 365 Government Cloud Community - GCC To meet the unique and evolving requirements of the United States Federal, State, Local, and Tribal governments, as well as contractors holding or processing data on behalf of the US Government, Microsoft offers the Office 365 Government GCC environment. Available through multiple channels including Volume Licensing, interested organizations go through a validation process to ensure eligibility before an environment is established. Trials are available to only US Government entities currently. Page 32 of 2498 • Microsoft 365 App Copilots (Word, Excel, PowerPoint): Embedded within your favorite productivity apps, these intelligent assistants help you craft documents, analyze data, and create presentations faster and with greater precision. • Outlook (Mail and Calendar): Manage communications and interagency meetings efficiently. Copilot in Outlook streamlines email and calendar management by drafting responses, summarizing email threads, and suggesting optimal meeting times. • Teams Chat/Channel Copilot: Enhance collaboration across departments. Integrate Al into your daily team interactions, summarizing threads, and helping manage conversations with contextual awareness. • Intelligent Meeting Recap: Automatically capture key points from important policy discussions or public hearings, capturing crucial details facilitating transparent governance. The Intelligent Meeting Recap feature generates a comprehensive summary of your meetings, including key points, action items, and decisions made. Page 33 of 2498 • Teams Meeting Copilot • OneNote • Stream • Planner • OneDrive for Business • Loop Page 34 of 2498 Proposed Implementation Timeline Policy Drafted IT and Pilot User Testing December 2024 November 2024 Licensing January 2025 Enterprise Licensing including Training, Governance and General Use March 2025 Pilot Use Cases 2025 and beyond July 2025 On -Going Attention to: BEAPS - Bias ,Equity, Accuracy, Privacy, Security Page 35 of 2498 40 Questions Page 36 of 2498