Evaluation for Popular Online Public MQTT Broker

Many MQTT projects and IoT services provide ​​online public MQTT broker​​. Users can directly use them for MQTT learning, testing, prototyping, and even small-scale applications without self-deployment, convenient and fast with time and energy saved.

However, due to different locations, network environments, and server loads, each public broker’s stability and message transmission delay are also different. Although almost all service providers declare that they are not responsible for the stability and security of their free services, users need to consider these factors when using them.

Therefore, this article sorts out some popular free online MQTT brokers, evaluates and compares them through accessibility, network delay, small-scale performance testing, and actual message transmission delay. We hope to provide a reference for your choice.

Test object

This article selects the following commonly used public MQTT brokers:

EMQX

It is a free online ​​MQTT 5 broker​​​ provided by ​​EMQX Cloud​​. There are two access points provided, Global and CN, of which EMQX (Global) is deployed in AWS, Oregon, USA, and EMQX (CN) is deployed in Tencent Cloud, Shanghai.

Both access points are EMQX clusters composed of 2 nodes. Later, more nodes can be automatically added according to the actual access volume and load. According to the background display, the server is based on ​​EMQX Enterprise​​ 4.2.6, and the current running time is 128 days.

Note: The two access points of EMQX (Global) and EMQX (CN) do not communicate with each other.

Related introduction: ​​Free Public MQTT 5 Broker​​

Eclipse

It is a free online MQTT broker provided by Eclipse IoT. The resolved IP shows that it is deployed in Azure, Virginia, USA. It’s worth noting that the previous access address of the server has always been ​​mqtt.eclipse.org​​. For some reason, it has been changed to the current access address. At the time of writing this article, I used the old address and failed to access it. At one time, I thought the server had been stopped. Finally, when I accessed the original access point through HTTP, I found that 301 permanent redirection had been made.

According to the query of the ​​$SYS/#​​ system topic, the server is based on the Mosquitto 2.0.12, and the current running time is 71227 seconds. It is suspected that the service was restarted a day ago.

Related introduction: ​​mqtt.eclipseprojects.io​​

Mosquitto

It is a free online MQTT broker provided by the Mosquitto community, and the resolved IP shows that it is deployed in the OVH, Roubaix region of French. In the test, it was found that the network delay of the access point is relatively high under normal conditions. Still, fortunately, the packet loss rate is relatively low, and connection failures may occur in some periods.

According to the query of the ​​$SYS/#​​ system topic, the server is based on the Mosquitto 2.0.12, and the current running time is 28519 seconds. It is suspected that the service was restarted within one day.

Related introduction: ​​test.mosquitto.org​​

HiveMQ

It is a free online MQTT broker provided by HiveMQ, and the resolved IP shows that it is deployed in the AWS, Frankfurt, Germany.

Because its ​​$SYS/#​​ system topic cannot be subscribed, it is impossible to get the type of Broker providing the service, the specific version, and the current running time.

Related introduction: ​​mqtt-dashboard.com​​

Test environment

• Network: US, AWS Oregon
• Operating system: macOS 10.15.7

Note: Due to different locations, the network environment will be different, which may lead to differences in the test results of this article.

Accessibility test

In this part, We used the ​​MQTT client tool - MQTT X​​ for this test. We try to establish a connection through TCP 1883. After repeated testing, all servers can be accessed. The overall results are as follows:

Test Data

MQTT X has the function of connection, import and export. The following is the connection data used in the test of this article, which can be imported into MQTT X through data recovery.

​​MQTTX-backup-free-public-mqtt-broker.json​​

International network delay test

We check the network connectivity and network delay through network access. Because some services have disabled the ICMP protocol, and the network conditions in various places are different, the WebSocket address is used here. With the help of the HTTP speed function of the popular domestic speed measurement tool Webmaster Tools, we conduct the test:

Test result

• Resolution time: the time required for DNS resolution
• Connection time: the time required to establish a TCP connection

Small-scale performance test

With the help of the open-source MQTT performance test tool ​​emqtt-bench​​​, we test whether the client's ​​Pub/Sub​​ has a rate limit.

For practicality considerations, this test is not to explore the upper rate limit of each access point, but to consider that each access point can meet the conventional use intensity. The scenario designed in this round is that a single client’s Sub/Pub message is 1000 msg/s for 1 minute, and the message size is 256 Bytes. Then, we record whether each access point meets the standard and whether the speed is limited. The following figure shows the test architecture:

After emqtt-bench is ready, each of the following Sub/Pub commands is executed in a different window:

# EMQX
## Sub
./emqtt_bench sub -t t/1 -c 1 -h broker.emqx.io
## Pub
./emqtt_bench pub -t t/1 -c 1 -h broker.emqx.io -I 1

# EMQX CN
## Sub
./emqtt_bench sub -t t/1 -c 1 -h broker-cn.emqx.io
## Pub
./emqtt_bench pub -t t/1 -c 1 -h broker-cn.emqx.io -I 1

# Eclipse
## Sub
./emqtt_bench sub -t t/1 -c 1 -h mqtt.eclipseprojects.io
## Pub
./emqtt_bench pub -t t/1 -c 1 -h mqtt.eclipseprojects.io -I 1


# Mosquitto
## Sub
./emqtt_bench sub -t t/1 -c 1 -h test.mosquitto.org
## Pub
./emqtt_bench pub -t t/1 -c 1 -h test.mosquitto.org -I 1

# HiveMQ
## Sub
./emqtt_bench sub -t t/1 -c 1 -h broker.hivemq.com
## Pub
./emqtt_bench pub -t t/1 -c 1 -h broker.hivemq.com -I 1

Actual message transmission delay test

Purpose: Test the time required for the message to transmit from the Pub to the Sub, sample and analyze the transmission stability and average time required.

Test step: The client connects to the public server and publishes a time-stamped message every 5 seconds. After the subscriber receives the message, the timestamp in the message is subtracted from the current timestamp, and the message delay is calculated and recorded in the database. Sampling and analysis are performed after 30 minutes of statistics.

The test model is as follows:

Test code: ​​free-online-public-broker-test.js​​

Time delay history

Count the client message delay and remove data with large errors (>5000ms):

Average delay

Summary

In several tests, each free online MQTT server has reached a usable level as a whole. However, there are still significant differences between the servers in the specific indicators. There are stability and usability issues like low rate limit, unstable network delay, or even some servers are suspected of having a scheduled restart mechanism, which will bring a bad experience to users even in simple testing and prototyping.

The above content also proves to a certain extent that the relevant performance of the IoT platform is affected by the geographic location of the device. Therefore, the EMQX free online MQTT service that provides nearby access points for users in different regions based on the network of high-quality cloud service providers has certain advantages in comparison, and the test data is relatively leading in all aspects.

We are delighted to see that more and more IoT devices worldwide are connected to the online MQTT server provided by EMQX, and thousands of messages are delivered per second on average. ​​broker.emqx.io:1883​​​ also appears in various open-source projects and sample codes on GitHub (​​https://github.com/search?q=broker.emqx.io&type=Code​​).

EMQX online public broker is provided by ​​EMQX Cloud​​. EMQX Cloud is a fully managed cloud-native MQTT messaging service provided by EMQ, which supports commercial-level accessibility and stability. Business users can quickly start projects at zero cost and implement MQTT device access in a simple and fast way by using EMQX Cloud. In the later stage, it can be expanded as needed according to the business development. Users can create access points nearby worldwide and enjoy the 7 * 24 technical support provided by the EMQ professional team.

Whether it is a personal or corporate project, EMQ is committed to providing the most appropriate MQTT messaging service for all kinds of users. If you have any comments or questions when using EMQX, please feel free to give feedback to our team at any time.