mirror of
https://github.com/openimsdk/open-im-server.git
synced 2026-04-28 06:19:20 +08:00
feat: Enhancements to OpenIM Engineering Practices with Standardizer and Tool Versioning (#2159)
* feat: add standardizer optimize makefile * feat: add standardizer optimize makefile * feat: add openim test docs * feat: add openim kafka docs * feat: add openim kafka docs * feat: add openim kafka docs
This commit is contained in:
@@ -0,0 +1,162 @@
|
||||
# OpenIM Kafka Guide
|
||||
|
||||
This document aims to provide a set of concise guidelines to help you quickly install and use Kafka through Docker Compose.
|
||||
|
||||
## Installing Kafka
|
||||
|
||||
With the Docker Compose script provided by OpenIM, you can easily install Kafka. Use the following command to start Kafka:
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
After executing this command, Kafka will be installed and started. You can confirm the Kafka container is running with the following command:
|
||||
|
||||
```bash
|
||||
docker ps | grep kafka
|
||||
```
|
||||
|
||||
The output of this command, as shown below, displays the status information of the Kafka container:
|
||||
|
||||
```
|
||||
be416b5a0851 bitnami/kafka:3.5.1 "/opt/bitnami/script…" 3 days ago Up 2 days 9092/tcp, 0.0.0.0:19094->9094/tcp, :::19094->9094/tcp kafka
|
||||
```
|
||||
|
||||
### References
|
||||
|
||||
- Official Docker installation documentation: [Click here](http://events.jianshu.io/p/b60afa35303a)
|
||||
- Detailed installation guide: [Tutorial on Towards Data Science](https://towardsdatascience.com/how-to-install-apache-kafka-using-docker-the-easy-way-4ceb00817d8b)
|
||||
|
||||
## Using Kafka
|
||||
|
||||
### Entering the Kafka Container
|
||||
|
||||
To execute Kafka commands, you first need to enter the Kafka container. Use the following command:
|
||||
|
||||
```bash
|
||||
docker exec -it kafka bash
|
||||
```
|
||||
|
||||
### Kafka Command Tools
|
||||
|
||||
Inside the Kafka container, you can use various command-line tools to manage Kafka. These tools include but are not limited to:
|
||||
|
||||
- `kafka-topics.sh`: For creating, deleting, listing, or altering topics.
|
||||
- `kafka-console-producer.sh`: Allows sending messages to a specified topic from the command line.
|
||||
- `kafka-console-consumer.sh`: Allows reading messages from the command line, with the ability to specify topics.
|
||||
- `kafka-consumer-groups.sh`: For managing consumer group information.
|
||||
|
||||
### Kafka Client Tool Installation
|
||||
|
||||
For easier Kafka management, you can install Kafka client tools. If you installed Kafka through OpenIM's Docker Compose, you can install the Kafka client tools with the following command:
|
||||
|
||||
```bash
|
||||
make install.kafkactl
|
||||
```
|
||||
|
||||
### Automatic Topic Creation
|
||||
|
||||
When installing Kafka through OpenIM's Docker Compose method, OpenIM automatically creates the following topics:
|
||||
|
||||
- `latestMsgToRedis`
|
||||
- `msgToPush`
|
||||
- `offlineMsgToMongoMysql`
|
||||
|
||||
These topics are created using the `scripts/create-topic.sh` script. The script waits for Kafka to be ready before executing the commands to create topics:
|
||||
|
||||
```bash
|
||||
# Wait for Kafka to be ready
|
||||
until /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092; do
|
||||
echo "Waiting for Kafka to be ready..."
|
||||
sleep 2
|
||||
done
|
||||
|
||||
# Create topics
|
||||
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic latestMsgToRedis
|
||||
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic msgToPush
|
||||
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic offlineMsgToMongoMysql
|
||||
|
||||
echo "Topics created."
|
||||
```
|
||||
|
||||
The optimized and expanded documentation further details some basic commands for operations inside the Kafka container, as well as basic commands for managing Kafka using `kafkactl`. Here is a more detailed guide.
|
||||
|
||||
|
||||
## Basic Commands in the Kafka Container
|
||||
|
||||
### Listing Topics
|
||||
|
||||
To list all existing topics, you can use the following command:
|
||||
|
||||
```bash
|
||||
kafka-topics.sh --list --bootstrap-server localhost:9092
|
||||
```
|
||||
|
||||
### Creating a New Topic
|
||||
|
||||
When creating a new topic, you can specify the number of partitions and the replication factor. Here is the command to create a new topic:
|
||||
|
||||
```bash
|
||||
kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic your_topic_name
|
||||
```
|
||||
|
||||
### Producing Messages
|
||||
|
||||
To send messages to a specific topic, you can use the producer command. The following command prompts you to enter messages, which are sent to the specified topic with each press of the Enter key:
|
||||
|
||||
```bash
|
||||
kafka-console-producer.sh --broker-list localhost:9092 --topic your_topic_name
|
||||
```
|
||||
|
||||
### Consuming Messages
|
||||
|
||||
To read messages from a specific topic, you can use the consumer command. The following command reads new messages from the specified topic and outputs them on the console:
|
||||
|
||||
```bash
|
||||
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic your_topic_name --from-beginning
|
||||
```
|
||||
|
||||
The `
|
||||
|
||||
--from-beginning` parameter reads messages from the beginning of the topic. If this parameter is omitted, only new messages will be read.
|
||||
|
||||
|
||||
## Basic Commands Using `kafkactl`
|
||||
|
||||
`kafkactl` is a command-line tool for managing and operating Kafka clusters. It offers a more modern way to interact with Kafka.
|
||||
|
||||
### Listing Topics
|
||||
|
||||
To list all topics, you can use:
|
||||
|
||||
```bash
|
||||
kafkactl get topics
|
||||
```
|
||||
|
||||
### Creating a New Topic
|
||||
|
||||
To create a new topic with `kafkactl`, use:
|
||||
|
||||
```bash
|
||||
kafkactl create topic your_topic_name --partitions 1 --replication-factor 1
|
||||
```
|
||||
|
||||
### Producing Messages
|
||||
|
||||
To send messages to a topic, you can use:
|
||||
|
||||
```bash
|
||||
kafkactl produce your_topic_name --value "your message"
|
||||
```
|
||||
|
||||
Here, `"your message"` is the content of the message you want to send.
|
||||
|
||||
### Consuming Messages
|
||||
|
||||
To consume messages from a topic, use:
|
||||
|
||||
```bash
|
||||
kafkactl consume your_topic_name --from-beginning
|
||||
```
|
||||
|
||||
Again, the `--from-beginning` parameter will start consuming messages from the beginning of the topic. If you do not wish to start from the beginning, you can omit this parameter.
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Script Logging Documentation Link
|
||||
|
||||
If you wish to view the script's logging documentation, you can click on this link: [Logging Documentation](https://github.com/OpenIMSDK/Open-IM-Server/blob/main/docs/contrib/bash-log.md).
|
||||
If you wish to view the script's logging documentation, you can click on this link: [Logging Documentation](https://github.com/openimsdk/open-im-server/blob/main/docs/contrib/bash-log.md).
|
||||
|
||||
Below is the documentation for logging and error handling in the OpenIM Go project.
|
||||
|
||||
|
||||
+83
-3
@@ -2,14 +2,94 @@
|
||||
|
||||
This document serves as a comprehensive guide to understanding and utilizing the `test.sh` script for testing OpenIM RPC services. The `test.sh` script is a collection of bash functions designed to test various aspects of the OpenIM RPC services, ensuring that each part of the API is functioning as expected.
|
||||
|
||||
+ Scripts:https://github.com/OpenIMSDK/Open-IM-Server/tree/main/scripts/install/test.sh
|
||||
+ Scripts:https://github.com/openimsdk/open-im-server/tree/main/scripts/install/test.sh
|
||||
|
||||
For some complex, bulky functional tests, performance tests, and various e2e tests, We are all in the current warehouse to https://github.com/OpenIMSDK/Open-IM-Server/tree/main/test or https://github.com/openim-sigs/test-infra directory In the.
|
||||
For some complex, bulky functional tests, performance tests, and various e2e tests, We are all in the current warehouse to https://github.com/openimsdk/open-im-server/tree/main/test or https://github.com/openim-sigs/test-infra directory In the.
|
||||
|
||||
+ About OpenIM Feature [Test Docs](https://docs.google.com/spreadsheets/d/1zELWkwxgOOZ7u5pmYCqqaFnvZy2SVajv/edit?usp=sharing&ouid=103266350914914783293&rtpof=true&sd=true)
|
||||
|
||||
## Util Test
|
||||
|
||||
## Usage
|
||||
Let's restructure and enhance the document under a unified second-level heading, adding clarity and details for better comprehension and visual appeal.
|
||||
|
||||
---
|
||||
|
||||
## Development Guide
|
||||
|
||||
### Comprehensive Testing Instructions
|
||||
|
||||
#### Running Unit Tests
|
||||
|
||||
- **Command**: To execute unit tests, input the following in your terminal:
|
||||
```
|
||||
make test
|
||||
```
|
||||
|
||||
#### Evaluating Test Coverage
|
||||
|
||||
- **Overview**: It's crucial to assess how much of your code is covered by tests.
|
||||
- **Command**:
|
||||
```bash
|
||||
make cover
|
||||
```
|
||||
This command generates a report detailing the percentage of your code tested, ensuring adherence to quality standards.
|
||||
|
||||
#### Conducting API Tests
|
||||
|
||||
- **Purpose**: API tests validate the interaction and functionality of your application's interfaces.
|
||||
- **How to Run**:
|
||||
```
|
||||
make test-api
|
||||
```
|
||||
Use this to check the integrity and reliability of your API endpoints.
|
||||
|
||||
#### End-to-End (E2E) Testing
|
||||
|
||||
- **Scope**: E2E tests simulate real-user scenarios from start to finish.
|
||||
- **Execution**:
|
||||
```
|
||||
make test-e2e
|
||||
```
|
||||
This comprehensive testing ensures your application performs as expected in real-world situations.
|
||||
|
||||
### Crafting Unit Test Cases
|
||||
|
||||
#### Setup for Test Case Generation
|
||||
|
||||
- **Installation**: Install the `gotests` tool to generate test cases automatically.
|
||||
```bash
|
||||
make install.gotests
|
||||
```
|
||||
This command installs the `gotests` tool for test case generation.
|
||||
|
||||
- **Environment Preparation**: Define your test template environment variable and generate test cases as shown below:
|
||||
```bash
|
||||
export GOTESTS_TEMPLATE=testify
|
||||
gotests -i -w -only keyFunc .
|
||||
```
|
||||
This prepares your environment for test case generation using the `testify` template.
|
||||
|
||||
#### Isolating Function Tests
|
||||
|
||||
- **Single Function Testing**: When you need to focus on testing a single function for detailed examination.
|
||||
- **Method**:
|
||||
```bash
|
||||
go test -v -run TestKeyFunc
|
||||
```
|
||||
This command specifically runs tests for `TestKeyFunc`, allowing targeted debugging and validation.
|
||||
|
||||
### Important Note
|
||||
|
||||
- **Quality Assurance**: Throughout your development process, it is imperative to ensure that the unit test coverage meets or surpasses the standards set by OpenIM.
|
||||
- **Maintaining Standards**: Regularly running your tests with
|
||||
```make test```
|
||||
supports maintaining high code quality and adherence to OpenIM's rigorous testing benchmarks.
|
||||
|
||||
## E2E Test
|
||||
|
||||
TODO
|
||||
|
||||
## Api Test
|
||||
|
||||
The `test.sh` script is located within the `./scripts/install/` directory of the OpenIM service's codebase. To use the script, navigate to this directory from your terminal:
|
||||
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 17 KiB |
|
Before Width: | Height: | Size: 207 KiB After Width: | Height: | Size: 207 KiB |
|
Before Width: | Height: | Size: 118 KiB After Width: | Height: | Size: 118 KiB |
Reference in New Issue
Block a user