Skip to content

Commit 3a5c2c6

Browse files
authored
Queue Statistics (#2)
1. Expose 6 queue metrics using micrometer. (queue-size, delay queue size, processing queue size, dead letter queue size, execution counter, failure counter) 2. Fix an issue in scheduler that's getting scheduled at the delay of 5 seconds. [this leads to messages are not copied from delayed queue to main queue on high load. 3. An api to move messages from dead letter queue to other queue. (Any source queue to target queue).
1 parent 3d818f3 commit 3a5c2c6

121 files changed

Lines changed: 4083 additions & 1302 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.travis.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ services:
88
env:
99
ORG_GRADLE_PROJECT_sonatypeUsername=xxx
1010
ORG_GRADLE_PROJECT_sonatypePassword=xxx
11+
USER_NAME=rqueue
1112

1213
cache:
1314
directories:

README.md

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ Rqueue is an asynchronous task executor(worker) built for spring framework based
1212

1313
### Adding a task
1414
Rqueue supports two types of tasks.
15-
1. Execute method as soon as possible
16-
2. Delayed tasks (task that would be scheduled at given delay)
15+
1. Execute tasks as soon as possible
16+
2. Delayed tasks (task that would be scheduled at given delay, run task at 2:30PM)
1717

1818

1919
### Task execution configuration
@@ -33,14 +33,14 @@ Add Dependency
3333
* Get latest one from [Maven central](https://search.maven.org/search?q=g:com.github.sonus21%20AND%20a:rqueue-spring-boot-starter)
3434
* Gradle
3535
```groovy
36-
implementation 'com.github.sonus21:rqueue-spring-boot-starter:1.2-RELEASE'
36+
implementation 'com.github.sonus21:rqueue-spring-boot-starter:1.3-RELEASE'
3737
```
3838
* Maven
3939
```xml
4040
<dependency>
4141
<groupId>com.github.sonus21</groupId>
4242
<artifactId>rqueue-spring-boot-starter</artifactId>
43-
<version>1.2-RELEASE</version>
43+
<version>1.3-RELEASE</version>
4444
<type>pom</type>
4545
</dependency>
4646
```
@@ -51,14 +51,14 @@ Add Dependency
5151
Get latest one from [Maven central](https://search.maven.org/search?q=g:com.github.sonus21%20AND%20a:rqueue-spring)
5252
* Gradle
5353
```groovy
54-
implementation 'com.github.sonus21:rqueue-spring:1.2-RELEASE'
54+
implementation 'com.github.sonus21:rqueue-spring:1.3-RELEASE'
5555
```
5656
* Maven
5757
```xml
5858
<dependency>
5959
<groupId>com.github.sonus21</groupId>
6060
<artifactId>rqueue-spring</artifactId>
61-
<version>1.2-RELEASE</version>
61+
<version>1.3-RELEASE</version>
6262
<type>pom</type>
6363
</dependency>
6464
```
@@ -242,6 +242,40 @@ factory.setMessageConverters(messageConverters);
242242

243243
More than one message converter can be used as well, when more than one message converters are provided then they are used in the order, whichever returns **non null** value is used.
244244

245+
## Monitoring Queue Statistics
246+
NOTE: Rqueue support micrometer library for monitoring.
247+
248+
**It provides 4 types gauge of metrics.**
249+
```
250+
1. queue.size : number of tasks to be run
251+
2. dead.letter.queue.size : number of tasks in the dead letter queue
252+
3. delayed.queue.size : number of tasks scheduled for later time, it's an approximate number, since some tasks might not have moved to be processed despite best efforts
253+
4. processing.queue.size : number of tasks are being processed. It's also an approximate number due to retry and tasks acknowledgements.
254+
```
255+
256+
**Execution and failure counters can be enabled (by default this is disabled).**
257+
258+
We need to set count.execution and count.failure fields of RqueueMetricsProperties
259+
```
260+
1. execution.count
261+
2. failure.count
262+
```
263+
264+
All these metrics are tagged
265+
266+
**Spring Boot Application**
267+
1. Add micrometer and the exporter dependencies
268+
2. Set tags if any using `rqueue.metrics.tags.<name> = <value>`
269+
3. Enable counting features using `rqueue.metrics.count.execution=true`, `rqueue.metrics.count.failure=true`
270+
271+
**Spring Application**
272+
273+
1. Add micrometer and the exporter dependencies provide MeterRegistry as bean
274+
2. Provide bean of RqueueMetricsProperties, in this bean set all the required fields.
275+
276+
277+
[![Grafana Dashboard](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/grafana-dashboard.png)](https://raw.githubusercontent.com/sonus21/rqueue/master/docs/static/grafana-dashboard.png)
278+
245279

246280

247281
## Support

build.gradle

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
plugins {
22
id 'jacoco'
33
id 'com.github.kt3k.coveralls' version '2.6.3'
4+
id 'nebula.optional-base' version '5.0.2'
45
}
56
allprojects {
67
apply plugin: 'idea'
@@ -9,6 +10,7 @@ allprojects {
910
apply plugin: 'maven-publish'
1011
apply plugin: 'signing'
1112
apply plugin: 'jacoco'
13+
apply plugin: 'nebula.optional-base'
1214
sourceCompatibility = 1.8
1315
targetCompatibility = 1.8
1416

@@ -20,7 +22,7 @@ allprojects {
2022

2123
subprojects {
2224
group = 'com.github.sonus21'
23-
version = '1.2-SNAPSHOT'
25+
version = '1.3-RELEASE'
2426

2527
dependencies {
2628
// https://mvnrepository.com/artifact/org.springframework/spring-messaging
@@ -52,7 +54,7 @@ subprojects {
5254
}
5355

5456
def publishedProjects = subprojects.findAll({ subproject ->
55-
subproject.pluginManager.hasPlugin('java') && !subproject.name.endsWith("example")
57+
subproject.pluginManager.hasPlugin('java') && !subproject.name.endsWith("example") && !subproject.name.contains("test")
5658
})
5759

5860
task jacocoMerge(type: JacocoMerge) {

docs/static/grafana-dashboard.png

354 KB
Loading

gradle/code-signing.gradle

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,8 +14,7 @@ artifacts {
1414
archives javadocJar
1515
}
1616

17-
18-
ext.isReleaseVersion = !version.endsWith("SNAPSHOT")
17+
ext.isReleaseVersion = !version.endsWith("SNAPSHOT") && !"rqueue".equals(System.getenv("USER_NAME"))
1918
signing {
2019
required { isReleaseVersion && gradle.taskGraph.hasTask("uploadArchives") }
2120
sign configurations.archives
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
FROM tomcat:8.0.51-jre8-alpine
2+
RUN rm -rf /usr/local/tomcat/webapps/*
3+
ARG WAR_FILE=build/libs/*.war
4+
COPY ${WAR_FILE} /usr/local/tomcat/webapps/ROOT.war
5+
CMD ["catalina.sh","run"]

0 commit comments

Comments
 (0)