Frank Carius

Author: Frank Carius
Enterprise Architect / Partner – auf LinkedIn vernetzen

For many organizations, implementing Microsoft Copilot is a challenge that goes beyond software integration. In addition to the employee transition, your network infrastructure needs to be well prepared. A good understanding of the processes is important to get the most out of Microsoft Copilot and to make your network secure and efficient.

In this article, we highlight the most important aspects of Microsoft Copilot’s network requirements. This will give you a deeper insight into the technical details required for a successful integration.

Microsoft Copilot, ChatGPT and other AIs: What are the differences?

Machine learning and “AI” in general are no longer niche topics. ChatGPT has become an integral part of everyday life for many companies, and Microsoft Copilot has recently become more than just a household name. Many companies are currently thinking about implementing Microsoft Copilot or are already using the service to make their own data more efficient. Although the underlying technology behind both tools is similar, Copilot and ChatGPT have different requirements:

  • ChatGPT und Co
    Microsoft, as well as Google and others, have used knowledge from the Internet and other sources to train their models and allow users to solve their tasks by asking questions. The AI relies on huge amounts of data to generate answers, images, music and the like. Once learned, the data is available to a broad group of users and is not individualized.

  • Microsoft Copilot
    In contrast, Copilot uses your own data, such as voice in Teams, documents in SharePoint or OneDrive, email in Exchange, and other sources from your tenant or data you provide, to learn and answer your questions. This is also reflected in the cost and minimum number of licenses.

  • Eigene KI
    Of course you can install your own AI locally and index your local servers independently from ChatGPT and Copilot.

Adapting Your Network for Microsoft Copilot

But what do you need to consider in your network infrastructure when deploying Microsoft Copilot? In general, Microsoft Copilot runs in the cloud and users access the various services in the background through your network. Microsoft has described the requirements for Microsoft Copilot for Microsoft 365 and the communication channels.

The article summarizes that Copilot first addresses the classic Microsoft cloud endpoints, so the same principles that apply to other Microsoft 365 services must be followed:

  • 1

    Local Breakout
    Avoid network latency by routing traffic directly to the Internet instead of detouring through the corporate LAN. It is advisable to process and filter traffic as close to the source as possible to increase efficiency.

  • 2

    Do not allow proxy servers to inspect the traffic of certain applications. This avoids delays caused by inspecting each packet and allows for more direct and faster access.

  • 3

    For home office workers, it is advisable to enable a direct connection without redirection through the corporate network. This not only reduces the load on the VPN and the company’s internal network infrastructure, but also improves user access times.

All these requirements are no different from the previous requirements for using Microsoft 365 and are already covered by network monitoring tools such as Rimscout. Incidentally, as of March 2024, Copilot in the Microsoft 365 apps also seems to check the certificate and refuses to analyze it with Fiddler, for example. This means that an inspection proxy with its own certificates is also likely to be a problem, and a workaround for these URLs will be required.

How do WebSockets work and what does it mean for your network?

A new requirement is that “WebSocket” connections must also be possible to a range of IP addresses listed at position 46 of the list ( instead of classic HTTP requests. This includes all URLs named “*, *,” as well as many other IP addresses.

Traditional Web site access is via HTTP/HTTPS, where a browser or client requests a URL using a GET method or sends data to the cloud using a POST method. Outlook asks for new email and the Exchange server responds immediately or when a new email is received. WebSockets have a different purpose in that they keep the connection open bidirectionally. They typically start with an HTTP GET, but then perform an “UPGRADE” to WebSockets if the remote peer supports it. A transparent tunnel is then established between the client and the server, through which both endpoints can send data at any time. WebSockets are therefore often used for services that send data to the client continuously or with minimal delay. For example, it would be very inefficient for the client to poll every second.

However, switching to WebSockets means configuring your network accordingly, as proxy servers and firewalls need to be set up to handle this change in behavior, since the connection can last a very long time and the transmission is “atypical” compared to HTTP traffic. Intrusion detection systems (IDS) or firewalls may block the connection.

Technically, it is still an HTTP/HTTPS connection that starts with a GET or POST and then changes mode within the protocol. The client makes an HTTP request to the WebSocket endpoint:

GET /chathub HTTP/1.1
Upgrade: WebSocket
Connection: Upgrade

The other side must respond with a “101 switching”:

HTTP/1.1 101 Switching Protocols
Upgrade: WebSocket
Connection: Upgrade

You will then see the request and the structure of each response in the WebSocket stream.

Rimscout and WebSocket Testing

Due to the way WebSockets work via HTTP/HTTPS, you can include the remote stations in a test with Rimscout and determine the important factors:

  • Configuration
    Rimscout can capture, report and check the local configuration of the client to detect basic errors such as incorrect DNS or proxy servers.

  • Accessibility
    Rimscout can use various test types (DNS, ICMP UDP, TCP, http, etc.) to check the general accessibility with a single test.

Rimscout can use various test types (DNS, ICMP UDP, TCP, http, etc.) to check the general accessibility with a single test.
However, in order to test the WebSocket feature itself, Rimscout would not only need to upgrade the connection from HTTPS to WebSockets, but also be able to simulate the WebSocket data stream. However, Microsoft Copilot and other services will always require authentication. This raises the question of the security of such credentials and the added value of such a thorough test.

From our point of view, it is sufficient to determine the quality of the connection to the Copilot service. We check if a WebSocket connection is possible, e.g. even if it is routed through an HTTP proxy.

Other components in Microsoft Copilot

Most sources only handle the communication between the user and the Microsoft Copilot backend to ask questions and receive answers. In addition, Microsoft Copilot can also query external data sources or include them in the semantic index. There are two ways to do this:

  • 1

    This feature connects the Copilot search to other data sources. For example, as an administrator, you can include file servers, Web sites, and other services. Microsoft Copilot connects to the services to add the content to the “semantic index”. This process is asynchronous and therefore not time-critical, but can involve large amounts of data.

  • 2

    This extension to Microsoft Copilot on the client uses the user’s query to send it to other internal services and enrich it with the results. For example, a connection to an ERP/CRM system or a helpdesk can be realized.

Both connections communicate with the service over the network. However, Microsoft Copilot is the initiator and not the client, so a network monitor such as Rimscout cannot record any values here.

To sum up

The demands placed on the network by user queries to Microsoft Copilot are not critical. This is not real-time traffic, and the AI usually needs more time to parse the request into tokens, query the semantic index, and generate the results than the latency in the network would be noticeable. However, the connection must always be available and must not be blocked by proxy servers or similar.

Try Rimscout in your environment.
Is your network ready for Microsoft Copilot? Try Rimscout’s full functionality for one month free of charge.