Microsoft Copilot details: What you need to know now about network requirements

For many organizations, implementing Microsoft Copilot is a challenge that goes beyond software integration. In addition to the employee transition, your network infrastructure needs to be well prepared. A good understanding of the processes is important to get the most out of Microsoft Copilot and to make your network secure and efficient.

In this article, we highlight the most important aspects of Microsoft Copilot’s network requirements. This will give you a deeper insight into the technical details required for a successful integration.

Microsoft Copilot, ChatGPT and other AIs: What are the differences?

Machine learning and “AI” in general are no longer niche topics. ChatGPT has become an integral part of everyday life for many companies, and Microsoft Copilot has recently become more than just a household name. Many companies are currently thinking about implementing Microsoft Copilot or are already using the service to make their own data more efficient. Although the underlying technology behind both tools is similar, Copilot and ChatGPT have different requirements:

Adapting Your Network for Microsoft Copilot

But what do you need to consider in your network infrastructure when deploying Microsoft Copilot? In general, Microsoft Copilot runs in the cloud and users access the various services in the background through your network. Microsoft has described the requirements for Microsoft Copilot for Microsoft 365 and the communication channels.

The article summarizes that Copilot first addresses the classic Microsoft cloud endpoints, so the same principles that apply to other Microsoft 365 services must be followed:

All these requirements are no different from the previous requirements for using Microsoft 365 and are already covered by network monitoring tools such as Rimscout. Incidentally, as of March 2024, Copilot in the Microsoft 365 apps also seems to check the certificate and refuses to analyze it with Fiddler, for example. This means that an inspection proxy with its own certificates is also likely to be a problem, and a workaround for these URLs will be required.

How do WebSockets work and what does it mean for your network?

A new requirement is that “WebSocket” connections must also be possible to a range of IP addresses listed at position 46 of the list (https://aka.ms/o365ip) instead of classic HTTP requests. This includes all URLs named “*.officeapps.live.com, *.online.office.com, office.live.com” as well as many other IP addresses.

Traditional Web site access is via HTTP/HTTPS, where a browser or client requests a URL using a GET method or sends data to the cloud using a POST method. Outlook asks for new email and the Exchange server responds immediately or when a new email is received. WebSockets have a different purpose in that they keep the connection open bidirectionally. They typically start with an HTTP GET, but then perform an “UPGRADE” to WebSockets if the remote peer supports it. A transparent tunnel is then established between the client and the server, through which both endpoints can send data at any time. WebSockets are therefore often used for services that send data to the client continuously or with minimal delay. For example, it would be very inefficient for the client to poll every second.

However, switching to WebSockets means configuring your network accordingly, as proxy servers and firewalls need to be set up to handle this change in behavior, since the connection can last a very long time and the transmission is “atypical” compared to HTTP traffic. Intrusion detection systems (IDS) or firewalls may block the connection.

Technically, it is still an HTTP/HTTPS connection that starts with a GET or POST and then changes mode within the protocol. The client makes an HTTP request to the WebSocket endpoint:

GET /chathub HTTP/1.1
Host: copilot.microsoft.om
Upgrade: WebSocket
Connection: Upgrade

The other side must respond with a “101 switching”:

HTTP/1.1 101 Switching Protocols
Upgrade: WebSocket
Connection: Upgrade

You will then see the request and the structure of each response in the WebSocket stream.

Rimscout and WebSocket Testing

Due to the way WebSockets work via HTTP/HTTPS, you can include the remote stations in a test with Rimscout and determine the important factors:

Rimscout can use various test types (DNS, ICMP UDP, TCP, http, etc.) to check the general accessibility with a single test.
However, in order to test the WebSocket feature itself, Rimscout would not only need to upgrade the connection from HTTPS to WebSockets, but also be able to simulate the WebSocket data stream. However, Microsoft Copilot and other services will always require authentication. This raises the question of the security of such credentials and the added value of such a thorough test.

From our point of view, it is sufficient to determine the quality of the connection to the Copilot service. We check if a WebSocket connection is possible, e.g. even if it is routed through an HTTP proxy.

Other components in Microsoft Copilot

Most sources only handle the communication between the user and the Microsoft Copilot backend to ask questions and receive answers. In addition, Microsoft Copilot can also query external data sources or include them in the semantic index. There are two ways to do this:

Both connections communicate with the service over the network. However, Microsoft Copilot is the initiator and not the client, so a network monitor such as Rimscout cannot record any values here.

To sum up

The demands placed on the network by user queries to Microsoft Copilot are not critical. This is not real-time traffic, and the AI usually needs more time to parse the request into tokens, query the semantic index, and generate the results than the latency in the network would be noticeable. However, the connection must always be available and must not be blocked by proxy servers or similar.