-
Anonymous users2024-02-15Summary. If the consumption rate of Kafka is slow, you can try the following methods to tune it1Adjusting the Number of Consumer Instances Based on the Number of Topic PartitionsAs we all know, each partition of a topic can be consumed by only one instance in the consumer group in order to ensure the order of message consumption.
If the number of partitions on your topic is 36, you can try to adjust the number of consumer instances to 36, of course, this is the case of consumer = 1. If you adjust the number of threads to n, your optimal number of consumer instances is. 2.
If you enable Bulk Consumption, you can enable Bulk Consumption and set the number of Batch Consumption = 100, which is 5003 by defaultConsumers consume data in batches: Multithreaded processing: Consumers use thread pools to consume data in batches. It is recommended not to use execute, otherwise it is easy to get out of control, it is recommended to use submit and then get, and when get, it will wait for the thread to be executed, and this way will consume the batch and then consume it down.
If the consumption rate of Kafka is slow, you can try the following methods to tune it1Adjusting the Number of Consumer Instances Based on the Number of Topic PartitionsAs we all know, each partition of a topic can be consumed by only one instance in the consumer group in order to ensure the order of message consumption. If the number of partitions on your topic is 36, you can try to adjust the number of consumer cases to 36, of course, this is the case of consumer = 1.
If you adjust the number of threads to n, your optimal number of consumer instances is. 2.If you enable Bulk Consumption, you can enable Bulk Consumption and set the number of Batch Consumption = 100, which is 5003 by default
Consumers consume data in batches: Multithreaded processing: Consumers use thread pools to consume data in batches. It is recommended not to use execute, otherwise it is easy to get out of control, it is recommended to use submit and then get, when get, it will laugh and wait for the thread to be executed, this way will consume the batch of this batch of consumption before consuming it down.
In this case, the number of core threads in the thread pool needs to be determined according to whether the business is IO-intensive or CPU-intensive.
-
Anonymous users2024-02-14at most onece modeThe basic idea is to ensure that each message is successfully committed, and then consumed.
Set the automatic imitation of hidden ants to false, and after receiving the message, it will be committed first, and then consumed.
at least onece mode.
The basic idea is to ensure that each message is successfully processed before committing.
Set auto-commit to false; After the message is processed, manually commit.
When adopting this model, it is best to ensure the "idempotency" of consumption operations to prevent repeated consumption.
Exactly onece mode.
The core idea is to handle the OFFSET as a unique ID at the same time as the message, and to ensure the atomicity of the processing.
Set the auto-withdrawal deficit to false; Submit the message after it is successfully processed.
For example, for a buried relational database, you can set the id to the unique index of the message processing result, and if the index already exists when it is processed again, it will not be processed.
Today's advertising has been like a flood of all the best spaces, and consumers who seem to be full of information cannot get real and reliable information resources to help them distinguish between high-quality and fake products and services. This is where the customer report was born. The Customer Reporting Platform is a professional, independent, and not-for-profit organization. >>>More
The basic characteristics of the consumer market are as follows: >>>More
When we configure the computer or upgrade the computer, the choice or replacement of the graphics card is a very important thing, because the performance of the graphics card is directly linked to the game experience, a good graphics card can bring a better gaming experience, but the better the performance of the graphics card, the more expensive, so how to choose the right graphics card for yourself is very important. <> >>>More
Consumer attitudes refer to consumers' emotional responses to objects, attributes, and interests, that is, consumers' tendency to respond to a product, brand, or company that they have learned to like or dislike consistently. There are two views on the composition of attitudes: one is monism, which holds that attitudes are composed of affective elements, that is, attitudes are understood only as an emotion or emotion; The other is triadism, which holds that attitudes are composed of three elements: cognition, emotion, and intention.
If two wait operations, i.e. wait(full) and wait(mutex), swap places, or signal(mutex) and signal(full) swap positions, when the buffer is full of k products. The producer has produced another product that will wait on the empty when it wants to deposit it in the buffer, but it already has the right to use the buffer. At this time, when the consumer wants to take the product, he will stay on the mutex and not get the right to use the buffer, resulting in the producer waiting for the consumer to take the product, while the consumer is waiting for the producer to release the right to use the buffer, and this mutual waiting will never end. >>>More