Beyond Copilot: Building Enterprise AI Agents That Actually Work with Copilot Studio with Manpreet Singh [MVP-MCT]

Enterprise AI is moving beyond chatbots that simply answer questions. The next generation of AI agents can understand business context, connect to enterprise systems, orchestrate workflows, and take action on behalf of users.But building an impressive AI agent demo is easy. Building an agent that works reliably across a global enterprise—with sensitive data, complex business processes, governance requirements, thousands of users, and measurable outcomes—is a very different challenge.In this episode of the M365 FM Podcast, Mirko Peters talks with Manpreet Singh [MVP/MCT] about what it actually takes to build production-ready enterprise AI agents with Microsoft Copilot Studio and the wider Microsoft AI ecosystem.Manpreet explains how organizations are evolving from individual departmental agents toward multi-agent architectures where specialized agents collaborate behind a single interface. The discussion explores when to use Microsoft 365 Copilot, Copilot Studio, and Azure AI Foundry—and how these technologies can work together rather than becoming isolated AI platforms.

FROM ANSWERS TO ACTIONS
The real transformation begins when an AI agent can do more than retrieve information.Using practical examples, Manpreet explains how agents can connect with systems such as Workday, Salesforce, SAP, ServiceNow, Jira, Confluence, and PeopleSoft. Instead of navigating several applications manually, employees can interact with an agent through Microsoft Teams or another conversational interface.An employee requesting leave, for example, could have an agent check available leave, consult HR policies, initiate the request in the underlying HR system, ask the manager for approval, and return the final result—all without the employee opening the individual applications.

KNOWLEDGE, TOOLS AND TRIGGERS
A useful enterprise agent requires more than a good prompt.Manpreet breaks the architecture down into essential elements: knowledge sources that provide organizational context, tools and connectors that allow the agent to interact with enterprise systems, and triggers that determine when processes should begin.SharePoint can become an important knowledge layer, while Power Platform connectors and APIs enable agents to perform actions across business applications.

WHY DATA QUALITY MATTERS
Connecting an agent to twenty years of SharePoint content is not necessarily a good strategy.Manpreet strongly recommends cleaning and curating organizational knowledge before exposing it to AI. Thousands of outdated PDFs, duplicate documents, missing metadata, and obsolete policies can undermine the quality of agent responses.A smaller, carefully maintained knowledge base with current documents, useful metadata, tags, descriptions, and version management can produce significantly better results than simply indexing everything an organization owns.

HUMAN-IN-THE-LOOP AI
Autonomous does not have to mean uncontrolled.For low-risk transactions, organizations may allow an agent to complete an action automatically. Higher-value or sensitive decisions can introduce human approval.Manpreet discusses examples including financial claims, invoice processing, access requests, and infrastructure changes where humans remain part of the decision-making process while AI handles much of the repetitive work surrounding the decision.

GOVERNANCE BEFORE SCALE
As agents gain access to multiple enterprise applications, governance becomes critical.The conversation explores Microsoft Purview, Data Loss Prevention policies, security controls, sensitivity labels, Microsoft Defender, identity, environment strategies, and the importance of establishing an AI Center of Excellence before allowing agent development to expand throughout an organization.Governance should protect enterprise information without creating policies so restrictive that agent performance and usability suffer.

CONTROLLING AGENT SPRAW

Welcome everybody back to a new episode of the MC65FM podcast.

For the last few years, Enterprise AI has largely been about one thing, ask AI a question

and get answer, but we are now entering a very different phase.

AI is moving from answering questions to understanding business context, connecting

to enterprise systems or casting workflows and increasingly taking action on behalf of

users.

In other words, we are moving from co-pilot to agents, but building an impressive

agent demo, it's relatively easy, build an agent that works reliable inside a global enterprise

with real users, real business processes, sensitive data, security requirements, governance,

integrations and measurable business outcomes, it's something different, definitely.

My guest today is Manfred Sein, Microsoft MVP and NCT and leader of the Modern Workplace

AI Solutions team at Cochneycent.

Modern works with global enterprise across industries, including financial service, healthcare,

media and entertainment, designing Microsoft-based AI solutions and the agenteic AI strategies.

He also shared his technical knowledge with millions of readers and organized the boot

camps, hackathons, conferences and other community initiatives around AI and Microsoft

technologies.

Today we are going beyond the co-pilot type.

We are going deep into Microsoft co-pilot studio enterprise agent, autonomous AI, multi-agent

architecture, governance and so on.

So yeah, welcome on Fritusha, thank you so much for being here.

Thank you, well, thank you for the invite.

I'm so excited to be here and being part of your amazing podcast.

Thank you, thank you.

Before we deep dive into the agent world, so tell us a little bit about your journey into

the Microsoft technology.

Oh, yeah, yeah, of course.

So I think I spent my whole career life with Microsoft tech and Blackforks, started my career

with .NET and then went to SharePoint, SharePoint, more, 32 on 7, then 2030, 2060, SharePoint,

online power platform, power automated, and now I saw about AI, right?

That's where I came in and now I have been supporting multiple world-years, multiple clients

where there are insurance, healthcare, life size, bank, anywhere, helping them and sustaining

and building these AI solutions for them.

Yeah, cool.

So yeah, direct deep dive in the world, traditional co-pilot, interacts of like summarise this, write

this, find this information, what change when we move towards solve this problem for me?

Right.

So I think the earlier before the world AI hit, we were all working on SaaS products, whether

it's Microsoft Suite 365, power platform, and we were very happy with the world, right?

And then all of a sudden, charge-gbd announced, I think, I was one of those fun boys, this is

what it can do, it's so very, it was really something way beyond, right?

We all remember charge-gbd through that process, adding a little step time or something.

And then slowly we started moving towards a braiding agent, a pirate agent, personal agent,

organization agents, companies started coming in, they like, one could any 10 agents, one for

each department, and then started building the catalog of agents.

And then slowly it became multi agent, where companies are like, I just need one agent and

a child agent, right?

So and it kept evolving, even now there are MCPs, the prototypes, configurations, which

people are going with multi agent, multi LLM models coming in picture, they started evolving.

That's where now our job, we come in and like me and you, we tell them, we talk about AI

and how AI is coming in, helping in, go, we start here, we're talking about how it can

make your life, the end their life easier, your employee experience as an end user is much

better, it's not just depending on SaaS product, where you're opening portals and doing things.

Now agent is doing some of the things for you and building that whole end-to-end solution,

that's what AI is capable of.

Every day something or the other, whether it's loud or jam, or even Microsoft, they have

so many announcements, something or the other is coming in, organizations are immediately

adopting to it and scaling with it.

That's the future of AI and end-up solutions today.

Yeah.

Let's design an AI agent.

How are the major architectural components of production and enterprise agent?

Right.

Let's take a client, for example, the client who was in Microsoft XTAR, my M365.

He or she will reach out here like, "Makpith, I want to start a journey in agent, right?

And I am an insurance company who deals with mortgage insurance auto came home, okay?

So how do we start from there, right?

So first thing would be, hey, what's your preferred XTAR?

And of course, we are talking about Microsoft, so they are like, okay, Microsoft would expect

preferred XTAR with the use cases, whether it's through between Microsoft 365, co-pilot,

which is the agent builder, no code, no code.

And co-pilot studio, which has become much more advanced with multi-agent solutions and

multi-agent multi-aller providers.

And then you have Azure AI Foundry where you're building these models much more designing,

much more in-depth, compared to other those two parts.

So you divide the use case here to three.

And the best part of Microsoft ecosystem is they talk to each other.

So it's not like Microsoft co-pilot is a standard agent and then studio agents are standard

on the all talk to each other.

They're all connected through teams and multiple, you know, of API connector.

So an example, as you talk about, right?

They somebody company will come, hey, I want to build an agent for my company.

You can call it as Jarvis.

My employee should be able to come in and do sub transactions.

Now, imagine the day when you have to apply for a leave before AI, you used to go to a

voltage system or a people's office system, check your leave balance and then go and apply

for a leave.

It used to trigger an email to your manager where he or she needs to approve it.

And then they will get a confirmation as your leave has been approved, right?

Now that has changed.

Now if I have to go for a leave on Monday, I just want to say, I'm going to do a leave

on Monday.

I just have to ask the agent, hey, agent, can you apply for a personal day off on Monday?

It understands what Monday is.

It understands what my personal day a leave is based on the categories of leaves I have.

Check my balance.

If I have balance immediately since a team's pop up to my manager, hey, if you any approve

or decline or share your comment, he or she approves it.

And I get a notification, hey, your leave is approved.

And everything was updated in the people's soft system in the voltage system.

So I didn't open any of those systems.

I just went to my co-pilot, chat, I had to phrase and like, hey, apply for a leave, right?

That's an example I gave.

Next one I'll tell you.

There are so many insurance calls that just contact centers, right?

They get so calls.

So if I call my insurance, hey, I want to see what my current interest rate is, right?

So as soon as I call, my agent is also part of the conversation.

And while I'm talking and I share my member ID, my member ID is 1001.

And goes get all information of their member ID and populates in the screen of the contact

center where a contact center usually will have to go search multiple systems in their

G-R, the box, and check my profile and test and my profile.

Now AI just gets all information.

Some rises it and help you and suggest, hey, this is what the new person there is.

And you can also give him a couple of more offers.

So it became my life faster.

The 10 minutes call now becomes a one minute to a minute call, you're saving eight minutes,

right?

That's the ROI.

So these are some of the agents which you can build and bring it to your ecosystem from as

simple as an HR policy agent or a leaf tracker system or service now, connect to the agent

or the health of the fight.

These small, small fixations, you start from co-pilot, co-pilot studio or Azure 8.

Or bring your own model, right?

You can extend a scale these solutions as such.

Awesome.

I think we have four or five things when we talk about the agent.

We have the instructions, we have the knowledge topic, we have the action and tool topic,

the reasoning topic.

How do we do it all fit together?

Sure, sure.

So let's take the same example, the holiday tracker, right?

I'm applying to leave.

So when I'm applying, he is an API call for validating and checking my balance.

So that's the simple API tool connector which I'm adding as a connector to work day,

which is readily available.

There's a plus part of Microsoft suite.

And I just can't do my API service account and I'm all set right.

Then second thing is the knowledge, right?

Now again, my best part is ecosystem share point.

We have all the HR lease policies, documents loaded in a share point size.

So if I need to understand, hey, can I apply more leaves than I have for I'm traveling to

Italy or some other, right?

So my HR system, the documents which are there in share point, which has no metadata,

but AI reached them and pro indexes started for questions.

And if you're an outward, yes, you can apply and take some leave from the next year portal,

right?

So that's where the knowledge comes in.

So you have the tools where you're applying reading access or editing the access to work

hand system, knowledge is where you are doing this.

And then the third part is the trigger, right?

What should be the trigger for me?

The trigger was I went to teams and I asked to apply for a leave for it.

So for other people, it could be an email, where I sent an email to a mailbox and the

mail boss understands the query, could kick off my agent and then apply for a leave, right?

So there's all the normal days.

And so is the these three pillars, the trigger, the tool set and the knowledge shows it could

be share point, it could be data set, it could be files, if you have directly uploaded,

all becomes an investor and your agent just need these three things to kick off and provide

their solution to you.

And I think especially when we talk about knowledge, most enterprise agents are accessing

to, yeah, or organization, how do you approach the grounding?

Of course, I tell the customer, please go and bring your 20 years of share point as an

index.

Please don't do that.

And I keep, I want to get that in the park, that's it as well.

What happens is like company, they like, hey, they'll, they'll build a co-parts, studio

agent and they'll add HR or IT sites as knowledge source, which has data from 1995 to 2026, right?

But the agent doesn't understand that, right?

It's very, very important.

As part of the governance, I tell, please clean up your data, make sure the data, which is

really it set or the latest data is only index to the data, so to talk agents because

always when they see those answers coming in, like, hey, when is the upcoming leave?

It can provide you a data from September 22 or 26 because it's not a prondent.

So you have to give a very clear restriction, provide me an upcoming leave in September,

26 and to avoid that and avoid user frustration.

Please make sure the data is cleaned up.

You only have the data which you want to index to the agent.

You will see how agent understands very quickly as part of the indexing data.

Right?

But please, please, please don't bring your heavy thoughts of 2020,000 files of taking a co-parts

to the agent.

It's born.

Okay.

So that's the, the part when the peer view comes into the game.

Yeah, because it's very, just to be honest, the metadata and the rat quality is very important

for you and fight.

So if you have done been 500 unstructured PDFs, I do a knowledge shows with no tags, no

description, nothing, no watch and control, you're just building a random answer generator.

It's, it's not gonna help you, right?

Until you tag them, make sure you retire or archive the old files and give them a new fresh

copy.

You should tell people why you're on a creative new library where you just put in your test

hard work 200 files and then shared with an agent, you will see the real experience, how

the differences between those 10,000 files, which is from 20 years, compared to your data

of rack metadata tag index files.

Awesome.

Yeah.

I think that when, when we build the agents and so on, but really, yeah, I think that,

the action begins when, yeah, when the, when AI handled something for you, right?

You say, knowledge is, is, is useful, but agents become much more interesting when they

can do something.

Like, what types of actions have you implementing, also, of the standard thing?

Or a lot, like, so as part of enterprise, we have connected to Salesforce, SAP, World Day,

people, software application, service, now, Gira, Confluence, a n number of providers,

right?

So right now, when you're building this enterprise agent, you don't want it to be only

stuck with Microsoft knowledge, say, is right?

You want them to give an enterprise agent, which works with all their systems, whether

they're a custom system or their tools of other company, then indication endpoint is very

important.

Copied studio and foundry, there's a best part, they already have those integrations built

from so many years, which is because of power, platform gateways and power, platform characters.

You just into them, bring them as part of the ecosystem.

And then when I as an end user, I'm conversing or doing an action, like applying a lead on

World Day, everything is being done on my teams or my agent interface, which is deployed

in a chain, trying to add a web app or whatever.

And it's calling multi agents, multi systems, multi tools and getting me done in form itself,

right?

Recently, we were working with one of the client, they'll just give you an example, right?

They have Salesforce, right?

So what they do is, so there are sales people who are always on the reward, all right?

And I like me and you met, like, hey, Merco is a nice person, and he works for this company.

Let me require his contact details, right?

Usually what will happen is you open Salesforce on your phone or an agent on, right?

And you will make an entry.

And now with the AI, the fast press AI, I just call, hey, I just met Merco.

This is his email ID.

He works as a CTO for this company, record, right?

And that just gets added his knowledge based on my sales for the environment for my whole

sales, seeing the normal work.

It was pretty, I just had to make a voice note instead of going and opening a SaaS program,

adding your information to that, right?

So those systems, those integrated systems is what help you scale the solutions and bring

that as part of your enterprise.

Yeah, I think how build we, build we stuff that's stopping, say, here, say, Merco, here's

what you add, but you add Tril2, I have every, I have done it for you.

From an enterprise, I take trip, perspective.

How can we do this?

Yeah, to just go.

Yeah, so first step is open up, right?

And when I speak at conferences, I always give them homework, right?

Sometimes we speak and sometimes it's too much for them.

So I create really simple steps as homework.

So homework number one, open co-piles studio, right?

Very simple, open co-piles studio.microsoft.com.

Just go ahead.

And if you don't have a license, you will get a trial version for 30 days and then you

can keep extending till 90 days.

Open that up.

First thing what you should do is create a blank agent, okay?

It's really a blank agent.

Call it Jarvis.

I'm an Iron Man fan.

All my agents are Jarvis, okay?

You put agent name as Jarvis.

Add a knowledge source.

Now the knowledge source could be your share point, but sometimes the company don't allow

you to do it if you don't have license.

Don't worry about it.

Add a web site from learn.microsoft.com or add your own company domain, whatever company

you're working for, Microsoft.com, com is in.com.

Add it as a knowledge source, okay?

So first thing is your knowledge source is ready.

Second thing is you want to set up a tool, like whenever I get an information from AI, send

that to me in an email, right?

Very easy.

So that is your trigger.

So whenever AI, agent, piss up your information, it's a trigger and it can record and send

it to your email.

That's the tool.

That's your action you can build.

If you have some time, you can also create an excel sheet like, hey, create this record and

add it to my excel as a row item.

Everything is there in Power Automate.

Add a new row item.

You choose that action.

You added the now when you're chatting with an agent, you are getting your grounded data

form learned out of my source.com.

It can build it as an academy for your company.

Or you can do these processes where you are recording data on an excel file or sending

it as an email.

So those action, those trigger and then knowledge source, anyone can start from day one.

That's what Hoback's story is very easy to start with for 30 days, 60 days.

And that's what we do at our workshop, right?

We give you that slag.

We give you that solution and you do step by step.

The first agent is difficult and then the second, third, you'll be becoming a pro because

these agent builder has become so simpler, so easy that you can connect these tools and

number of tools and number of your hand device systems and build it as one.

Awesome.

Yeah.

So what's with, we can do a lot of automatization, especially when we talk about agents, multi-agent

architecture and now we say or a lot of people speak about autonomous multi-agent agents.

So where did you see the human in the loop?

Right.

So human loop will always come when there's a cost based cycle.

For example, if you're working for a financial crime and there's a claim processing, if it's

a claim with AI justifies as a $50 or $30, I think most of the companies are allowing AI

to take that decision.

But if there's a claim, which is $10,000, right?

So they would want a human in the loop to uproot or decline this, right?

Even in, even in like a usual proposal, right, where there are invoices coming in, but there

used to be a team of 10 people who used to monitor each invoice and then scan it and upload

it in an IBM data cap or any other server.

And then there used to be a process automation to extract data and digitize it.

Right now, AI is doing that.

And there's just one person sitting behind the screen who just validates the invoice because

AI will, it's self-taile, if there is 100% accurate or 60%.

You as a human in the loop, you activate that and you do that transaction, whether it's

creating tickets, whether it's approving server requests or it's access requests, if you're

getting all those you require human in the loop.

And even keeping human in the loop has become so simpler.

You are either prompting them on an email, you're either prompting them on a telephone,

hey, you're giving them a call while the agent is processing an information or you're sending

them a text message, hey, this is where I am at, like cloud code, right?

For example, I have an habit, I keep dreaming.

So I dream something, I put it on the cloud code and it starts working for me.

As human in the loop, it keeps asking me, is it, okay, am I on the right direction?

I'll, yes, you are.

I'll, no, no, this is wrong.

It fixed it, right?

So that human in the loop will always be the quiet in AI because we are not replacing

human.

We are just making them more empowered at their work cycle.

The time they used to do certain activity, if you should take an hour or 30 minutes,

now it used, now it takes a minute or so.

So me as a human, I can do other stuff much more efficiently at faster than reading on

my lazy work.

So with the AI's, we are not replacing people, we are just empowering them to do their job

much better.

Awesome.

Yeah.

Yeah.

Let's, let's talk.

We are the human in the loop, I think it's one security aspect from, yeah.

But yeah, we have other things.

So what changes when an AI system can access multiple enterprise applications?

Yeah.

So see, because governance for AI can't be designed in a vacuum, you need to make sure

all of your systems are far out the same governance sector, right?

You learn by setting up those guardrails across the clients I work with, right?

Whether you're working for a solutions from Microsoft tech stack to Amazon tech stack to service

now, okay, they all come under the same purview, Microsoft security purview or DLP policies

are applied or the sensitive labor is applied across the system.

Some clients are here, I don't want any PHI information to be loaded to do it, right?

So you know them, all those as part of the CUE, you when you set up.

So that's why it's a client.

So you start buying agents from the market or start building agents.

First is set up your governance, setup CUE.

So the set of excellence for you to set up that governance is very important.

Now, when you set up this governance, also make sure you don't make it so tough for each

agent, like I'll give an example, somebody wanted to build an agent which was extracting data

from one drive and teeth, right?

So those two platform, but the DLP which was applied to them was from enterprise level,

working service now, there's and that like too many.

So the response was coming was like a minute, I was waiting for the response to come is coming

in a minute like why you have to design your environment, your DLP policy as such based

on what you're building, what AI it affidels your building is required.

If I'm building an agent who just require one drive and team, lock down your DLP policy

on that environment of that agent to just get me information from that.

Is there a blocking everything like, hey, I want to be a secured engine and lock, it will

not have you.

Make sure your agents are deployed in multiple environments, make sure it all has specific

DLP or per view or defenders connected to the AM and system.

And then most of your system, your enterprise is some which only existing whether it service

now, constantly, they're already in part of your enterprise governance.

The API is only the key which you're connecting through or MCP, my favorite.

I always love MCP to do it.

It's already covered in skate.

Just make sure your governance is set up before you bring these agents for your enterprise

level.

You don't want them to come and upload her passport copy or SS and copy.

You want to make sure you block it.

And there are so many tools which we have built where even when somebody uploads, it started

to, you get a blocker, you're uploading a page at for person, hold up.

So you can bring those practices as part of your day to day life cycle in your enterprise

and you will see how agent building is much more secure.

I come from a company, we have 350 K associates.

It's a lot of people globally, lots of people working in people in Germany, Australia, China,

UK.

Everyone has their own environment, everyone has their own DLP for your policy set.

So it's the governance is much more easier when you do all of that.

So please set up a COE first and then start rolling out your agents.

Yeah, I think when we talk about, we have multiple tools.

We have the Azure policy, we have the defender, we have peer view and imagine much

more.

Yeah, and body and so on.

How did all these tools fit into an enterprise ready, a architecture?

Yeah, so Microsoft now is making sure it's locking down the entire ID with defender and

purview.

All the policies are in sync.

So you don't, you're not creating multiple policy in purview, which is not reflecting

on your defender or start reflecting to your entry or security groups are in sync or your

active, your data, DLP, so in sync.

So it the enterprise modules of these systems are already working and I said, it's a set

in the being because Microsoft is a system.

It makes your life easier.

The same security groups, the same user groups that applied across your tenant, whereas

in Friday, outlaw, the SharePoint and bring these policies across organization across

regions.

Like I know you have the different sex of policy compared to what US and China had.

So it's very easy for seeking between all the security, operand and building the sustainable

systems.

Okay.

I think I heard from companies like yours, the company, some companies have thousands of

agents.

What's will I say, I don't know if it's, it's worth it, which so exists, but what is with

agent life cycle management?

Yeah.

Oh, that's the next, that's what we are doing.

So taking off a dock, the hospital, right, and hospital has a little doctor.

Someone is specialist in cardiology, somebody's specialist in here, knows ENT, somebody's for

boards, right?

So when, as I said, before the multi agent network came in, people with building agents,

I have companies were built like, yes, hundreds of thousands of agents for just small purposes.

But if they are building one agent for HR, it's trying to service no confidence and they

are making building another agent for IT, which is also connecting the service of confidence,

but they just want to call it as IT agent and HR agent.

But when multi agent that became easier for them now, those thousands of agents are getting

committed under a child relationship between the multiple agents.

So one agent drives another audit, another agent checks the output, working as a policy,

awesome.

I don't know, the kind of one agent is doing agent to agent calling or one agent is doing agent

to the system, the enterprise system, call it, right?

So that has become lower.

So now the multiple agents are rounding towards their platform and then you have eight

and 365 as dashboards, which are coming for the security productivity with dashboard.

What are the agents utilized?

How many agents have connected to what data sources?

How many agents are being used by external people?

That has become already in tune too.

And agent 365 is still there's a lot of potential is still just started off.

I think it came a couple of months back.

So we as consumers, we are bringing every single agent whether it's build on cloud, whether

it's below Microsoft, Blacksteer, you build them in agent 365.

That creates a whole loop for making sure there are no agents without owners, making sure

your agents are not connected to connectors which require additional access or environment

uprooled.

So that dashboards have started coming up.

Yes, it became a pain point and then Microsoft brought in a, this is the agent 365.

The future would be yes, the agents are monitoring itself.

There would be a security agent monitoring all the agents and hoping sure all the preview,

all the DLP defenders are applied to it.

But for now is a manual process.

You are adding agent like entriety.

We all have entriety principle.

Id is a spack in unique IDs agents now have those agent IDs agent and dry.

Where they get added to agent 365, they have a unique ID and you are able to monitor what

that agent is doing.

Yes, it is.

So, I think a little bit step in this side, we have this security co-pilot.

Yeah, but it's more monitoring, it's not doing something really from from, from, from,

quite so.

Yeah, yeah, that's a good feature.

It's free and admin center.

You can ask him, hey, check my license, check this.

So it's doing a basic work yet.

But yeah, I would want it to do everything in the security admin center.

Yeah, yeah.

I like, like, talk a little bit more about agent governance.

I thought this topic was really interesting.

So, I think for years we had shadow IT then shadows are then power platforms broad.

How are we adding towards again, towards again agents broad?

You, not yet.

I would say it will because I think what happened last year, the, the, the number of agents

started increasing in every single company, everyone was building some of the other purposes.

But now, as I said, they are controlling these agents as parent child relationship.

So, that, this making governance much more easier compared to those 10,000 agents now have

fewer agents, fewer covered, closely covered agents.

So, I don't think we are there yet.

This year, I've seen companies adopting the security in governance very seriously.

They're making sure, because last year, everyone was learning when they go, they were checking

enterprise, they were checking, why is it so slow, slow because of the TLP.

So, they revised the AI platform, the revised COE now and I feel the future would be much

nicer.

Yeah, however, there are open source agents which can do a lot of things like cloud and all

right, the new fable is so powerful and people use it because people can use it for so

many other multipurpose.

But I think for public access or enterprise access, you'll be still under government control,

whether if you are using Microsoft ecosystem, I'm not heard anything going on.

So, should we be fine?

Awesome.

Yeah, I think this would become the future.

I think, yeah.

And when we look, or I have to think about my question, but the first jump a little bit,

in three to another topic, it's often recommended by Microsoft show there be a central specialized

AI center of excellence that did we do we need it?

Oh, yeah, you need that, right?

So, that's where this new agent 365, that would be a center of excellent.

You are able to see all the tools, the agents, you're able to see all the workflows which

are connected to again, workflows, you're able to see what are the data sources, people

are consuming in the company.

So, I can check, oh, yeah, people are going to share point, conference service, and there's

only third party tools, like who brought this third party tool?

Let's not approve, right?

So, that kind of governance is very important, I think.

As I said, I think agent 365 coming the picture, still the step one, I should do a lot more

in the future.

I think probably at the night, content, they'll announce more features into it as part of

the ecosystem.

But yes, it is very important.

Me as an admin, I want to see a dashboard where my 350 K people are building thousands of

agents, but I should be able to monitor who is building both and what are they connecting

to, right?

What actions are they building?

What knowledge shows is they are building?

What FCP is they are connecting?

I would want to see that dashboard, yes.

So, that sounds really interesting.

Yeah, what would I have?

Should agents have a kind of risk classifications?

There are, again, when you build agents, right?

You build agents for enterprise systems to modeling processes, right?

Again, as I said, the agent processing is not very complicated as we suppose.

I don't see any risk for enterprise system because you are a governing them.

Your data is not going outside.

Your data is locked, you're a subscription, right?

So, you're not using public domain chat, GPD or co-pilot or Gemini, so it's all, you're

using enterprise agents.

You're already governed with your DLP policy, so I don't see any risk going on.

Though I've seen people by the build something on the cloud court and like, or get up, right?

They sometimes lose some data because they, they progn it in such a way.

Other than that, and that's like in human loop, right?

It's always there and it's getting fixed.

I still don't feel there's any risk or growth.

And it's part of the picture.

If your data is governed by companies like Microsoft, they assure you, right?

And then you prick story up.

So when AI came in and I was trying to pitch this Microsoft co-pilot to one of the life

sites, because medicine maker in the United States, they were, I want Microsoft lawyer to

come to my office, sign me and I can be meant that my knowledge, my data will not go outside

to train the model, right?

And Microsoft sent the lawyer to confidently sign that document.

Yes, your data is your data and it will not be used to train the other model.

So that confidence is what we required a couple of years back and now comes in very much

in a variation where they're fine AI as a rest job.

At least on the governance side job does a different debate with me and you can have, but

for enterprise, improving their experience is really doing a job.

Awesome.

Awesome.

I have seen, I was linked in my research.

You worked around the concept of a unified AI command center.

Yeah.

Yeah.

And what exactly is an AI command center?

So again, AI command center would be again, starting with agent 365 or environment set

up by it.

You build a portal where a request comes in the you're governing every step.

Like for example, I'm in a company company X, I'm on a unified command center would be

people requesting for an agent or requesting for a tool or requesting for an MCP right from

my command center where the admin approves or decline.

Yes.

Okay.

Go ahead.

I'm approving your access to go and connect to service now.

And this is the API.

It is API client.

I declined.

Right.

You automate that whole process where you are also doing auditing.

You are also doing validating.

You're also checking making sure everything is human in the loop.

That's where the command center, I would say, it would comment right and the command center

for you to is build an agent and then you see agent and where you are also able to monitor

the agent, how age this working is providing relevant information or is just blabbering

because there are 20,000 files which is connected to it.

So that is another part of the monitoring you want to see.

You want to see that the agent is being shared with right people right set.

Nobody is of doing a Bitcoin farming there right.

So you need to control that put some limit put some shared it.

Hey, don't go above a thousand dollar a month right.

For example, right.

What if somebody brings in a Bitcoin farm and starts doing on AI service and you get

a bill of 100 K. So you put limitations on each agent each environment on what how much

crop should go.

So that you can monitor when you get a flag, you reach $1000.

Why did they reach $1000?

My employees are just 100 who is doing what right.

So you need to bring those as part of the command center.

But again, as I said, COE is all part of your COE.

You need to make sure you set it up and then it meets your life as an enterprise much easier.

And I think from the admin perspective, it's also do is the or we have two roads.

We have the normal user who will build this app and say, okay, please give me $100 for

this.

I don't know.

And this access rights.

And then we have the the admins.

So is this also think for my idea is it's like an agent inventory tool for the company.

Yeah.

Yeah.

It is.

So yeah, we said the right thing.

So when copilot was a now right, the infrastructure required an agent builder people started

building thousands of agents, right.

And the admins are like, who does not understand AI.

They are good and administrator job, right.

But what are these saying is, how do I control them?

I don't know what they are building.

I see 1000 in a day coming in.

What the first step they did was they turned off the feature where you can build an agent.

They're like, they're like, they're like, they're not.

I don't know what to do.

I don't understand who is building the seasons and then get the list of where they're

going to do.

So that inventory was a very important factor.

Microsoft gave you power platform.

It added an agent feature there that dashboard started coming in, right.

Then it's still coming in.

They're not remotely yet.

So at least the admin was I, oh my God.

Okay.

I see all the agents in one place.

Okay.

Which is good.

And the next step, how do I see the dollar source or the tools that connect me.

So agent 365 now is doing the same purpose.

What an admin would have a shout it.

I've seen admin screaming at a conference to a person like, I want an admin center.

I want to see an inventory of all the agents, right.

That was missing.

But now I think people are happy that they were able to see inventory.

They're able to see understand all the dollars that tool set.

So as admin is very important.

They, they, they, they, they understand what is coming into the system.

It could be a cyber security access issue or DLP issue and future.

They need to make sure all API is at the world and controlled, which they're having doing

with multiple system are also being consumed by the AI agents.

Yeah.

So yes, they wanted that inventory and now they have it.

So they have very happy about that.

Okay.

Awesome.

Also, and it's also having a part of, I think, yeah, it's a lot, not the nicest topic,

but we are all interested about the token, getting more than experience.

It's, it's a dead dynamic.

What?

I think this time was trying to show the US dollars and all in.

And now we will pay for the token.

It's also be possible to have a cost overview with this.

Yeah.

I know.

It's a lot of cost.

I even, even for me is like, do you, I'm paying $30 for Microsoft corporate and you want me

to pay another four credits and corporate studio.

And now I think there's a bit of hardness there also charging there.

So this is a lot of quiet.

I think even to do a click, you would have to pay something, right?

But when it's, it's just because the AI is expensive right, the parking, the data centers

and data, what they are expensive at least.

So the more users use it, the possible radio, but the, and the usage is also increasing a lot

of other factors.

So those are the thing.

So yeah, I think right now, this very limited authentication in admin center where you can

control the credits that, hey, this agent should be only consuming these many credits.

I would want to see that I'm able to control that per user also so that I, that would help

me in that work.

Like me and you are pro users.

You can consume 10,000 prompts, right?

And what if there's another user who just wanted to come and check something and because I

consume everything of that agent.

Now that they are, they are getting, so you don't have enough credit, right?

Or enough corporate funds, right?

So I think, yes, the companies will start now understanding because right now they were

happy with the $30 corporate cost and then they were happy with corporate studio as being

the interface of building agents, but when you charge them for AI builder credits, you charge

them for power platform premium, panator.

And yes, they will want to see how much have I spent every month, every quarter, even

every day.

That is still missing because, yeah, it's not there, but I think that will come because people

are asking for it.

Is this more an idea or is such a product in development beta or it is a project in development?

Yeah, it is there.

It will come.

It has to come right now.

I'm able to see trends on power platform admin center and a little bit on the agents

65, but eventually yes, I would want to see.

So many requests have already come to me.

One thing can I check how many prompts is the agent is consumed or the credit, right?

What are the overall credit they want to understand?

What particular, again, what can be used?

They will want to see that.

One thing I think actually is, I find it's really, really interesting whether this, yeah,

with this AI command center and I have next days I have a live stream.

We build an enterprise agent and in the second step you do is for, yeah, for producers.

And it would be really cool if you ready.

We can test it on the talent tool.

I think that's, that's, that's, I really, I really, I really interested to see it.

That would be amazing.

Yes, yes, you build an agent in real time and then validated in the backend.

That would be really cool.

Yes.

Yeah.

And everyone in the book can test it, right?

How many fonts and how many message credit they are using?

Then in the next episode you must bring your tools, shall we also line?

That's cool.

Oh, I love that.

I love that.

Yeah.

Observability.

Traditional applications have logs.

What do observability mean for agente AI?

Again, again, it's, you've started with your metrics, right?

They want to see the ROI of for the agents and where they're picking up how many prompts are

getting returned with a real value out of it, right?

That is very important and observability be part of agent 65 now.

Microsoft is going very, very hard because as you said, right?

They want to know how the agents are inventories happening.

What are the people consuming across the system?

Observability is the next new thing.

Everywhere, if you see Microsoft frontier from how observability is being changed with agent

65 and then bring E7 as part of the license system where they're making sure you are connected

to your overview and defender systems as well.

So that's a new, I would say word, the key word in making sure that governance is in place

for all the areas.

Yeah.

Sure.

Should we log prompts from?

Okay.

I think here here, Germany, the most people say, oh, I don't get spied out.

But yeah, okay.

How do you, do you travel shoot a user reporting?

Yesterday, the agent didn't, something did something strange.

Why?

How can you look into it?

Yeah, I think how we can look into the process, but what these I agents do and how we can fix

it?

That's the observability, right?

What the agent is doing, right?

The traces were the trace, right?

Full chain of the reasoning or the call to action it did when I use a profit.

You want to understand what it did, what was the process, where it went, what was the knowledge

source, right?

You want to understand the metric sort of it and get that lost structure.

So when the data is accessed, what it was at access, what is access, PDF1 or what is

access PDF2 or the access PDF1 and two both, right?

So that's where right now when you call about microsoreo, they got new features as part

of where you see the transaction when you have typing on my test agent.

Hey, give me information about HR policy on taking a leave.

It goes, it gives me a whole chart where my agent is going to which share point or which

knowledge it actually answered at head and then I get a response from there.

So you get that whole training of picture coming up and this copies to the observability

really gives you a proper UI to make you sure how the trace of data is happening is also

there in Azure copies, observability also where you can see the what was the outcome with

the, and you have those thumbs up thumbs down, right?

People normally don't do that.

I always tell people, hey, if you got a right answer, you might have a question.

What's up?

Because agent understands the source was good, the tooling was correct, why it did the exact

good job, what it's supposed to do.

So that whole observability is where the monitoring, the automation and the governance will

come as part of the global picture.

Yeah, I think observability becomes so, it's so strange when you think there are agents

that work autonomously.

Yeah.

So, yeah, but I think also we have to talk about testing AI agents and how do you testing

something was answer our deterministic?

Oh, I love this new feature in copies, studio, the evaluation feature.

I don't know if you have use it or not.

It creates those test use cases, understanding what the agent capability is and then you put

that and stores it.

So, as I said, I want agent to do all the stuff, right?

And it's exactly doing that.

Agent creates those test use cases and you input them as data source and data injection

and it gives you an output how it did it to and then you keep creating those use cases

and the evaluation and keep feeding it to the agent provider.

You understand whether the data is being indexed properly or not.

Sometimes you feel the index didn't happen well.

So we go back in the SharePoint admin center, we indexed the SharePoint list or something

or the library, right?

We indexing the whole part of the picture.

So that will help and part of the structuring the data set and that helps in making sure

the agent, population, whatever the output is coming, you're getting the real output.

So the test is good.

Now this copies studio toolkit, I always tell people it's a free tool by the cat team.

You please please use it, you'll deploy that as your Microsoft app source and power platform

that helps you in creating and testing more use cases as part of your evaluations and all

those stuff.

So test, test, test your agent, make sure you're getting a proper output proper because those

testing also added to the prompt because you are training through prompts and through testing.

Your agents become much better rather than when you build an agent and give it to an end user,

the agent will perform by won't perform the way you want to.

So based on those test use cases, the evaluation, the, you know, learns on the go on the process.

Awesome.

But how did we test against, I say, how are the two nations?

Okay.

What is this?

Sorry, am I earning it?

You're pondering.

I think how, how I figure out if my AI starts to hallucinating.

Oh, yeah.

So again, the two options, top set thumbs up, that's the end user governance trail where they

can, they've written like the answer to thumbs up, but only evaluation for test cases like

what is the co-parts, you go to co-parts, you go to evaluation sector, you're feeding responses.

You go very detailed use cases in there are PDF features 105, 105, we pick up data from

each of them.

We create a user trail where you use a would be asking a similar kind of questions on those

document sets and we feed it to the agent and we validate how the agents look.

As I said, the first step is, make sure your data is clean, your data, where your indexing

is has proper metadata and tags and the description.

If you don't do that, then the response will be all hallucinated.

Most of the organization, they turn off the feature web features, like they don't want

data to come from the web.

They only want your data to come from the grounded data, which you are indexing.

So that minimizes your knowledge to go and hallucinate.

Okay.

So step one, don't use the web because web is like Google and Bing, right?

There's so much data in there right or wrong.

So you want to make sure your data, latest data, proper data, aligned data, structured data

is input to the agent and then agents coming.

Those were the days when the hallucination was very strong.

Now with new models in GBD and OPPO, the hallucination is very, very, very negative.

And then we often talk about this multi-agent system.

Have you an example how this looks like?

Is it like a normal organization structure?

I have a high R agent and that puts me, says, okay, I need to send the WallerPuzz and then

hire them.

How does it do the work?

Right.

Yeah.

I've seen company doing amazing demos.

I'll give you an example for somebody builds an agent, a multi-agent, like I am the CEO.

I will be the top one whole team, put a data and then the CEO has multiple departments,

HR, marketing, sales, and things like that.

Those become some multi-agent frameworks.

So when I ask, hey, can you create a marketing slide for me?

I'm meeting Marco for a podcast.

So it goes, it understands the requirement and the marketing agent has in my network goes

and picks up that task and starts working on it.

Whereas then the second word, hey, can you help me book a flight?

I have to go and meet Marco on next Wednesday in the person.

My other agent, my tracker or travel agent, who was an understanding, started looking for

a flight for me.

Tasty multi-agent network.

So early on it was happening, well, before multi-agent, you were doing one or one life.

I have to go travel, I have to go to travel agent.

If I have to go and HR agent for, I have to go to HR agent.

Now I just go to one agent, that agent.

Now people have fancy names in normalization.

Somebody calls them as Pepsi or somebody calls it as IWA, somebody calls it as some GPD or

fancy GPD names, IWA, LX.

So you just go to that agent, hey, do this for me.

That agent will become a multi-agent server.

And I get to fight that agent to agent to work for you instead of you looking for an agent

to do that work.

Hey, where is a travel agent?

Let me find, it's not a point of find, you don't do that.

You just go to your one agent and mix for us as Microsoft extract its copilot.

Go to copilot, copilot is already connected to multiple agents in the back end.

It will go and consume those agents or agents or those agents to providers and get your input

or your own, your love you for.

Something that I have seen on my research, your technical, your blog has reached more than

10 million readers.

How have you do it with an agent?

Yeah, yeah, yeah.

Those were the times when we used to write, right?

Yeah.

Like when we used to sit, when we used to learn.

And that's how I started.

I started by sharing knowledge through articles.

If I'm writing a PowerShell script, hey, I wrote a script, I'll put it on my blog or article,

okay, you can start using it from there.

Or if I'm doing some new snippet of a tool or I'm learning something, okay, I learned this.

Now, let's put it on the article.

I think that's where it came and then I made sure I made it easier for people to learn.

I want to complicate it because even for me, if somebody puts an article or a blog,

was like, just put a quote there, I will understand like, what does it do?

Like, all of a sudden, you give me step by step, step three, it goes to my mind, it visits

the other, that's how I got it.

Yeah.

Yeah.

Wow.

I could talk another hour with you.

That's really, I really enjoy it.

So I have every, every podcast, I have a rapid fire out.

So I, short, short, short, short answer.

So, uh, co-part, co-part, studio or custom code?

Power Automate or Autonomous Agents?

Power Automate.

Powerful agent or many specialized agents?

Many specializing.

Local or pro code?

Pro code.

Rack of fans, uni.

Right.

Uh, Shabot or Daita Wars?

Shaboi.

Uh, Shaboi Gain.

Yeah, uh, when, uh, the, your phone rings and such are the other calls you would say,

"Modern, you, you, you do a so great job.

Uh, I need you for the co-pilot studio and you can develop, uh, future you like, you get

all the money and resources.

What do you build?

To be honest, if, if I have to build an agent, I have many agents.

I have my own personal agents.

I have my phone and everything.

Right.

But right now, I, this year, I was thinking, uh, if I have to build an agent, I'm going

to build an agent, which will directly work with, uh, the climate change, the heat, the

fire, making sure, uh, get some renewable resources like water and all that it used to

be.

I think I want that agent, which we can go that kind of a geography, mapping and helping and,

and sharing those ideas for the governments around the world, maybe with the United Nations,

so that they can use the power of my agent and, and you can help them to bring, uh,

a life to a better peaceful, it used to be.

Yeah.

And, um, who should I invite next and what questions should I ask?

Uh, to the next guest.

Yeah.

Uh, and if it's on the Microsoft tech side, ask him or her like, um, what do you love more,

which agent are like more M665 co-pilot, GitHub co-pilot or co-factory agent?

And, and an idea.

Will I should ask?

Uh, no, I don't have an idea.

Yeah.

Yeah.

Then my final question, as I'm under, imagine I'm, I'm a CEO.

Okay.

I, I cannot imagine it, but really, but I tried.

Uh, of, of a global enterprise, we already have, uh, Microsoft's 65 co-pilot.

We are experimenting with co-pilot studio.

Different departments are building agents.

Which, um, which are more of SharePoint content, sensitive data, regulation requirements and

all this stuff.

And, uh, the executive, uh, now I was asking me, when do these AI investments actually start

transforming the business?

So if you're sitting there with me, um, how did we design the next three months?

Yeah.

So when people ask you about ROI, they, they just feel, uh, and I've seen that multiple CEOs

and CEOs I talked to, or they're like, Hey, I build this agent.

I have 10 agents.

I don't see people using it, right?

The major factor of those ROI is adoption.

I'll tell you why, right?

Uh, I know you are on time, but I'll work well to be ready to create.

So people are building agents, organizations are building agents, but as the consumer as

an end user, you just announced them on an outlook email.

Hey, this is a new agent.

This will do this and this and that's it.

Who is an end?

People like, like, I'll just give myself as an example and, uh, non-ID person, right?

I just come to the office for my legal work for some, I'm a legal author, right?

I don't care about your multiple agents until I know what should I do with that agent,

right?

So, uh, if somebody gives me an agent, one fifth, here's the agent.

I'm like, okay, what should I do with this agent?

Should I book my flight ticket?

Should I buy a brick coin or should I buy a new Audi car?

You have to give me instructions, right?

That is where the adoption is a very important factor for all companies.

Even if you build 10,000 agents and you were looking for an ROI, we need to train our associates.

Every single associate, not just the Toxie suite or not just the manager level, ever single

user.

Somebody, one of my, one of the CIO of a big bag told me one very good thing I was talking

to him is like, "My friend, I want to not just build agents.

I want to empower my people to use agents, right?

That's where the adoption comes in.

You need to make sure that Manpreet and the other person, everyone knows how to use an

agent.

Everyone knows what the outcome of that agent should be and they know what the input they

can provide.

So, it should be a day in a life of Manpreet versus day in a life of a lawyer or day in a

life of an HR.

Every thing should be covered.

The more the people, the hands on the gear.

Right now you and me, right?

You know you wait for podcasts and you're going to extract this information, AI, put articles

and things like that.

You know how to use AI.

But if there's a person who doesn't know how to use AI, they want to still write an article

that will listen to my podcast, they'll write line by line and they'll spend four hours.

You will do that in one minute.

So that adoption is ready.

ROI comes and I have seen company from 0% of buying an M365 corporate license for

8 to 10 months, no adoption rate.

And then when you do this adoption training, personal based training, one training, it has

increased to 98 to 99 percent.

That's the adoption goal you need to bring.

And that would be your ROI because your company started using and becoming much more experienced

with much more empowered with a new tech stack and started bringing that knowledge in your

day to day cycle.

So that's my ROI pitch.

Yeah, awesome.

Yom Arvett, thank you so many for joining me today.

For me, the big takeaway from this talk is that next phase of enterprise AI isn't simply

about creating better problems or adding another chatbot to, I don't know, Teams or something.

It's more moving from answers to outcomes, agents can connect knowledge, applications, workflows,

business processes.

It's a really amazing time.

And that also means the difficult part for enterprise technology doesn't disappear.

Identity matters, security matters, data quality matters, governance matters, architecture matters,

testing matters.

But the most important thing, what most, or what's the most importantly, it's the people

using the systems also matter.

Yeah.

And thank you for giving this view and you have to come, this product is ready to live

stream and show it to us.

So for all the listeners, you find my information on the podcast page from this episode.

And yeah, thank you again so many for spending our with me.

Oh, thank you, Marko.

Thank you for this invite and thank you for bringing stories like me and your podcasts.

You're doing an amazing job.

Thank you for hosting me today.

Yeah, thank you.

Bye.

Take care.

Bye.

(music fades)