diff --git a/README.md b/README.md index 10cd16e8ffd..532c0300d70 100644 --- a/README.md +++ b/README.md @@ -89,27 +89,6 @@ bytes = SpanBytesEncoder.JSON_V2.encode(span); Note: The above is just an example, most likely you'll want to use an existing tracing library like [Brave](https://github.com/openzipkin/brave) -## Storage Component -Zipkin includes a [StorageComponent](zipkin/src/main/java/zipkin2/storage/StorageComponent.java), used to store and query spans and -dependency links. This is used by the server and those making collectors, or span reporters. For this reason, storage -components have minimal dependencies, but most require Java 8+ - -Ex. -```java -// this won't create network connections -storage = ElasticsearchStorage.newBuilder() - .hosts(asList("http://myelastic:9200")).build(); - -// prepare a call -traceCall = storage.spanStore().getTrace("d3d200866a77cc59"); - -// execute it synchronously or asynchronously -trace = traceCall.execute(); - -// clean up any sessions, etc -storage.close(); -``` - ### In-Memory The [InMemoryStorage](zipkin-server#in-memory-storage) component is packaged in zipkin's core library. It is neither persistent, nor viable for realistic work loads. Its purpose @@ -129,11 +108,29 @@ Note: This store requires a [job to aggregate](https://github.com/openzipkin/zip ### Elasticsearch The [Elasticsearch](zipkin-server#elasticsearch-storage) component uses -Elasticsearch 5+ features, but is tested against Elasticsearch 6-7.x. +Elasticsearch 6+ features, but is tested against Elasticsearch 7-8.x. + +It stores the analyzed data after receiving it, supports daily indexing and storage of data, +and optimizes storage space by reducing indexes. + +Note: This store requires a [spark job](https://github.com/openzipkin/zipkin-dependencies) to aggregate dependency links. -It stores spans as Zipkin v2 json so that integration with other tools is -straightforward. To help with scale, this uses a combination of custom -and manually implemented indexing. +### SQLike Datasource +Supporting relational databases, both [Elasticsearch](#elasticsearch) and similar storage systems utilize +table structures to store data content at the day level. + +the following databases are supported: +1. MySQL: uses MySQL 5.x features, but is tested against MySQL 5.6. +2. PostgreSQL: used PostgreSQL 9+ features, but is tested against PostgreSQL 9.6. +3. H2: uses H2 1.4 features, but is tested against H2 1.4.199. + +Note: This store requires a [spark job](https://github.com/openzipkin/zipkin-dependencies) to aggregate dependency links. + +### BanyanDB +The [BanyanDB](zipkin-server#banyandb-storage-components) component uses BanyanDB 0.5.0+ features, but is tested against BanyanDB 0.5.0. + +BanyanDB expected to offer better compression rates, lesser memory usage, +and less CPU consumption similar to [Elasticsearch](#elasticsearch) data capacity scenarios. Note: This store requires a [spark job](https://github.com/openzipkin/zipkin-dependencies) to aggregate dependency links. @@ -147,35 +144,11 @@ default. Search primarily allows the trace list screen of the UI operate. * `GET /autocompleteValues?key=X` - Distinct values of Span.tags by key * `GET /traces` - Traces matching a query possibly including the above criteria - When search is disabled, traces can only be retrieved by ID -(`GET /trace/{traceId}`). Disabling search is only viable when there is +(`GET /trace/{traceId}` or `GET /traceMany?traceIds=X`). Disabling search is only viable when there is an alternative way to find trace IDs, such as logs. Disabling search can reduce storage costs or increase write throughput. -`StorageComponent.Builder.searchEnabled(false)` is implied when a zipkin -is run with the env variable `SEARCH_ENABLED=false`. - -### Legacy (v1) components -The following components are no longer encouraged, but exist to help aid -transition to supported ones. These are indicated as "v1" as they use -data layouts based on Zipkin's V1 Thrift model, as opposed to the -simpler v2 data model currently used. - -#### MySQL -The [MySQL v1](zipkin-storage/mysql-v1) component uses MySQL 5.6+ -features, but is tested against MariaDB 10.3. - -The schema was designed to be easy to understand and get started with; -it was not designed for performance. Ex spans fields are columns, so -you can perform ad-hoc queries using SQL. However, this component has -[known performance issues](https://github.com/openzipkin/zipkin/issues/1233): queries will eventually take seconds to return -if you put a lot of data into it. - -This store does not require a [job to aggregate](https://github.com/openzipkin/zipkin-dependencies) dependency links. -However, running the job will improve performance of dependencies -queries. - ## Running the server from source The [Zipkin server](zipkin-server) receives spans via HTTP POST and respond to queries from its UI. It can also run collectors, such as RabbitMQ or Kafka. @@ -183,10 +156,12 @@ from its UI. It can also run collectors, such as RabbitMQ or Kafka. To run the server from the currently checked out source, enter the following. JDK 11 is required to compile the source. ```bash +# Init and update all submodule +$ git submodule update --init --recursive # Build the server and also make its dependencies -$ ./mvnw -q --batch-mode -DskipTests --also-make -pl zipkin-server clean install +$ ./mvnw -q --batch-mode -DskipTests -Dcheckstyle.skip=true --also-make clean install # Run the server -$ java -jar ./zipkin-server/target/zipkin-server-*exec.jar +$ java -jar ./zipkin-server/server-starter/target/zipkin-server-starter*exec.jar ``` ## Artifacts diff --git a/docker/README.md b/docker/README.md index 61c1e2ef2b0..380e7af84bf 100644 --- a/docker/README.md +++ b/docker/README.md @@ -38,8 +38,8 @@ We also provide [example compose files](examples/README.md) that integrate colle such as Kafka or Elasticsearch. ## Configuration -Configuration is via environment variables, defined by [zipkin-server](https://github.com/openzipkin/zipkin/blob/master/zipkin-server/README.md). Notably, you'll want to look at the `STORAGE_TYPE` environment variables, which -include "cassandra", "mysql" and "elasticsearch". +Configuration is via environment variables, defined by [zipkin-server](https://github.com/openzipkin/zipkin/blob/master/zipkin-server/README.md). Notably, you'll want to look at the `ZIPKIN_STORAGE` environment variables, which +include "cassandra3", "mysql", "h2", "postgresql", "banyandb" and "elasticsearch". Note: the `openzipkin/zipkin-slim` image only supports "elasticsearch" storage. To use other storage types, you must use the main image `openzipkin/zipkin`. @@ -69,7 +69,7 @@ For example, to increase heap size, set `JAVA_OPTS` as shown in our [docker-comp For example, to add debug logging, set `command` as shown in our [docker-compose](examples/docker-compose.yml) file: ```yaml - command: --logging.level.zipkin2=DEBUG + command: -Dlog.level=DEBUG ``` ## Runtime user @@ -92,14 +92,13 @@ accordingly. Ex. If your link name is "storage" for an Elasticsearch container: ``` - ES_HOSTS=http://$STORAGE_PORT_9200_TCP_ADDR:9200 + ZIPKIN_STORAGE_ES_CLUSTER_NODES=$STORAGE_PORT_9200_TCP_ADDR:9200 ``` The above is mentioned only for historical reasons. The OpenZipkin community do not support Docker's deprecated container links. ### MySQL -If using an external MySQL server or image, ensure schema and other parameters match the [docs](https://github.com/openzipkin/zipkin/tree/master/zipkin-storage/mysql-v1#applying-the-schema). ## Building images diff --git a/zipkin-server/README.md b/zipkin-server/README.md index ea584e229af..857fb87b695 100644 --- a/zipkin-server/README.md +++ b/zipkin-server/README.md @@ -1,15 +1,14 @@ # zipkin-server -Zipkin Server is a Java 1.8+ service, packaged as an executable jar. +Zipkin Server is a Java 11+ service, packaged as an executable jar. Span storage and collectors are [configurable](#configuration). By default, storage is in-memory, the HTTP collector (POST /api/v2/spans endpoint) is enabled, and the server listens on port 9411. -Zipkin Server is implemented with [Armeria](https://github.com/line/armeria). While it uses [Spring Boot](http://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/) -internally, Zipkin Server should not be considered a normal Spring Boot application. +Zipkin Server is implemented with [Armeria](https://github.com/line/armeria) and based on the [SkyWalking v9](https://github.com/apache/skywalking) as core. ## Custom servers are not supported -By Custom servers we mean trying to use/embed `zipkin` as part of _an application you package_ (e.g. adding `zipkin-server` dependency to a Spring-boot application) instead of the packaged application we release. +By Custom servers we mean trying to use/embed `zipkin` as part of _an application you package_ (e.g. adding `zipkin-server` dependency to an application) instead of the packaged application we release. For proper usage, see the guides below. @@ -27,8 +26,8 @@ Once you've started, browse to http://your_host:9411 to find traces! ## Endpoints The following endpoints are defined under the base url http://your_host:9411 -* / - [UI](../zipkin-ui) -* /config.json - [Configuration for the UI](#configuration-for-the-ui) +* / - [UI](../zipkin-lens) +* /config.json - Configuration for the UI * /api/v2 - [API](https://zipkin.io/zipkin-api/#/) * /health - Returns 200 status if OK * /info - Provides the version of the running instance @@ -43,7 +42,7 @@ HTTP API via data conversion. This means you can still accept legacy data on new By default, all endpoints under `/api/v2` are configured to **allow** cross-origin requests. -This can be changed by modifying the property `zipkin.query.allowed-origins`. +This can be changed by modifying the property `query-zipkin.zipkin.allowedOrigins` or environment `ZIPKIN_QUERY_ALLOWED_ORIGINS`. For example, to allow CORS requests from `http://foo.bar.com`: @@ -57,26 +56,25 @@ See [Configuration](#configuration) for more about how Zipkin is configured. The [Zipkin API](https://zipkin.io/zipkin-api/#/default/get_services) does not include a parameter for how far back to look for service or span names. In order to prevent excessive load, service and span name queries are limited by -`QUERY_LOOKBACK`, which defaults to 24hrs (two daily buckets: one for +`ZIPKIN_QUERY_LOOKBACK`, which defaults to 24hrs (two daily buckets: one for today and one for yesterday) ## Logging By default, zipkin writes log messages to the console at INFO level and above. You can adjust -categories using the `logging.level.XXX` property. +categories using the `log.level` property. -For example, if you want to enable debug logging for all zipkin categories, you can start the server like so: +For example, if you want to enable debug logging, you can start the server like so: ```bash -$ java -jar zipkin.jar --logging.level.zipkin2=DEBUG +$ java -Dlog.level=DEBUG -jar zipkin.jar ``` See [Configuration](#configuration) for more about how Zipkin is configured. ### Advanced Logging Configuration -Under the covers, the server uses [Spring Boot - Logback integration](http://docs.spring.io/spring-boot/docs/current/reference/html/howto-logging.html#howto-configure-logback-for-logging). -For example, you can add `--logging.exception-conversion-word=%wEx{full}` to dump full stack traces -instead of truncated ones. +Under the covers, the server uses [Log4j2](https://logging.apache.org/log4j/2.x/). +For example, you can add `-Dlog4j.configurationFile=path/to/log4j2-config.xml` to customize the configuration file of logging. ## Metrics @@ -113,98 +111,110 @@ exposition [text format version 0.0.4](https://prometheus.io/docs/instrumenting/ Collector metrics are broken down by transport. The following are exported to the "/metrics" endpoint: -Metric | Description ---- | --- -counter.zipkin_collector.messages.$transport | cumulative messages received; should relate to messages reported by instrumented apps -counter.zipkin_collector.messages_dropped.$transport | cumulative messages dropped; reasons include client disconnects or malformed content -counter.zipkin_collector.bytes.$transport | cumulative message bytes -counter.zipkin_collector.spans.$transport | cumulative spans read; should relate to messages reported by instrumented apps -counter.zipkin_collector.spans_dropped.$transport | cumulative spans dropped; reasons include sampling or storage failures -gauge.zipkin_collector.message_spans.$transport | last count of spans in a message -gauge.zipkin_collector.message_bytes.$transport | last count of bytes in a message +Metric | Tags | Description +--- | --- | --- +trace_in_latency_bucket_bucket | sw_backend_instance=$instance_host_$instance_port, protocol=$transport, le=$second_bucket | The process latency histogram of trace data +trace_in_latency_sum | sw_backend_instance=$instance_host_$instance_port, protocol=$transport | The total process latency of trace data +trace_analysis_error_count | sw_backend_instance=$instance_host_$instance_port, protocol=$transport | The error number of trace analysis ## Configuration -We support ENV variable configuration, such as `STORAGE_TYPE=cassandra3`, as they are familiar to +We support ENV variable configuration, such as `ZIPKIN_STORAGE=cassandra3`, as they are familiar to administrators and easy to use in runtime environments such as Docker. -Here are the top-level configuration of Zipkin: -* `QUERY_PORT`: Listen port for the HTTP API and web UI; Defaults to 9411 -* `QUERY_ENABLED`: `false` disables the HTTP read endpoints under '/api/v2'. This also disables the -UI, as it relies on the API. If your only goal is to restrict search, use `SEARCH_ENABLED` instead. -Defaults to true -* `SEARCH_ENABLED`: `false` disables searching in the query API and any indexing or post-processing +Here are the top-level key configuration of Zipkin: +* `ZIPKIN_SERVICE_NAME_MAX_LENGTH`: Maximum length of a service name; Defaults to 70 +* `ZIPKIN_ENDPOINT_NAME_MAX_LENGTH`: Maximum length of an endpoint name; Defaults to 150 +* `ZIPKIN_CORE_RECORD_DATA_TTL`: How long to keep tracing data, in days; Defaults to 7 days +* `ZIPKIN_CORE_METRICS_DATA_TTL`: How long to keep metrics data(such as service names, span names), in days; Defaults to 7 days +* `ZIPKIN_SEARCHABLE_TAG_KEYS`: Defines a set of span tag keys which are searchable. Defaults to `http.method` +* `ZIPKIN_SAMPLE_RATE`: The trace sample rate precision is 0.0001, should be between 0 and 1. Defaults to 1 +* `ZIPKIN_SERVER_PORT`: Listen HTTP, gRPC port for HTTP API, web UI, etc. Defaults to 9411 +* `ZIPKIN_SEARCH_ENABLED`: `false` disables searching in the query API and any indexing or post-processing in the collector to support search. This does not disable the entire UI, as trace by ID and dependency queries still operate. Disable this when you use another service (such as logs) to find trace IDs. Defaults to true -* `QUERY_TIMEOUT`: Sets the hard timeout for query requests. Accepts any duration string (e.g., 100ms). -A value of 0 will disable the timeout completely. Defaults to 11s. -* `QUERY_LOG_LEVEL`: Log level written to the console; Defaults to INFO -* `QUERY_NAMES_MAX_AGE`: Controls the value of the `max-age` header zipkin-server responds with on +* `ZIPKIN_STORAGE`: Storage of the tracing data: one of `elasticsearch`, `h2`, `mysql`, `postgresql`, `banyandb`, `cassandra3` + +### HTTP Service + +The server provides multiple HTTP services, which by default use the same IP and port to provide services externally. +If each service is configured with a different port, a new HTTP service would be started and provided for that specific service. + +The following several HTTP services are availableļ¼š +* `ZIPKIN_QUERY_REST_PORT`: Listen HTTP port for HTTP API and web UI to other port, If less than or equal to 0, `ZIPKIN_SERVER_PORT` would be used. Defaults to -1 +* `ZIPKIN_QUERY_NAMES_MAX_AGE`: Controls the value of the `max-age` header zipkin-server responds with on http requests for autocompleted values in the UI (service names for example). Defaults to 300 seconds. -* `QUERY_LOOKBACK`: How many milliseconds queries can look back from endTs; Defaults to 24 hours (two daily buckets: one for today and one for yesterday) -* `STORAGE_TYPE`: SpanStore implementation: one of `mem`, `mysql`, `cassandra3`, `elasticsearch` -* `COLLECTOR_SAMPLE_RATE`: Percentage of traces to retain, defaults to always sample (1.0). -* `AUTOCOMPLETE_KEYS`: list of span tag keys which will be returned by the `/api/v2/autocompleteTags` endpoint; Tag keys should be comma separated e.g. "instance_id,user_id,env" -* `AUTOCOMPLETE_TTL`: How long in milliseconds to suppress calls to write the same autocomplete key/value pair. Default 3600000 (1 hr) - -### Configuration file overrides -Under the scenes, all configuration are managed by Spring Boot. This means that properties may also -be overridden by system properties or any other alternative [supported by Spring Boot](https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html). - -We use [yaml configuration](src/main/resources/zipkin-server-shared.yml) to bind shorter or more -idiomatic ENV variables to the Spring properties ultimately in use. While most users should only use -environment variables, some may desire a properties file approach to override settings. For example, -knowing we set `spring.config.name=zipkin-server`, Spring Boot will automatically look for a file -named `zipkin-server.properties` in the current directory, and the same properties we set in yaml -can be overridden that way. - -If you choose to use property-based configuration instead of ENV variables, you are choosing to -self-support your configuration. This means you'll use [Spring Boot documentation](https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html) -or [StackOverflow](https://stackoverflow.com/questions/tagged/spring-boot) to resolve concerns -related to property resolution as opposed to raising issues or using our chat support. We have to -mention this because configuration of Spring implies vast responsibility and our resources must be -conserved for Zipkin related tasks. +* `ZIPKIN_QUERY_LOOKBACK`: How many milliseconds queries can look back from endTs; Defaults to 24 hours (two daily buckets: one for today and one for yesterday) +* `ZIPKIN_QUERY_ZIPKIN`: `-` disables the HTTP read endpoints under '/api/v2'. This also disables the + UI, as it relies on the API. If your only goal is to restrict search, use `ZIPKIN_SEARCH_ENABLED` instead. + Defaults to `zipkin` + +### Configuration overrides + +All configurations are stored in [YAML file](server-starter/src/main/resources/application.yml), and the configuration settings can be modified through environment variables prior to launching. + +The contents in the configuration file are organized into three levels: +1. **Level 1**: Module name. This means that this module is active in running mode. +1. **Level 2**: Provider option list and provider selector. Available providers are listed here with a selector to indicate which one will actually take effect. If only one provider is listed, the `selector` is optional and can be omitted. +1. **Level 3**. Settings of the chosen provider. + +Example: + +```yaml +storage: + selector: ${ZIPKIN_STORAGE:h2} # the h2 storage will actually be activated, while the mysql storage takes no effect + h2: + properties: + jdbcUrl: ${ZIPKIN_STORAGE_H2_URL:jdbc:h2:mem:zipkin-db;DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=FALSE} + dataSource.user: ${ZIPKIN_STORAGE_H2_USER:sa} + metadataQueryMaxSize: ${ZIPKIN_STORAGE_H2_QUERY_MAX_SIZE:5000} + maxSizeOfBatchSql: ${ZIPKIN_STORAGE_MAX_SIZE_OF_BATCH_SQL:100} + asyncBatchPersistentPoolSize: ${ZIPKIN_STORAGE_ASYNC_BATCH_PERSISTENT_POOL_SIZE:1} + mysql: + properties: + jdbcUrl: ${ZIPKIN_JDBC_URL:"jdbc:mysql://localhost:3306/zipkin?rewriteBatchedStatements=true&allowMultiQueries=true"} + driverClassName: org.mariadb.jdbc.Driver + dataSource.user: ${ZIPKIN_DATA_SOURCE_USER:zipkin} + dataSource.password: ${ZIPKIN_DATA_SOURCE_PASSWORD:zipkin} + # other configurations +``` + +1. **`storage`** is the module. +1. **`selector`** selects one out of all providers listed below. The unselected ones take no effect as if they were deleted. +1. **`default`** is the default implementor of the core module. +1. `driver`, `url`, ... `metadataQueryMaxSize` are all setting items of the implementor. + +At the same time, there are two types of modules: required and optional. The required modules provide the skeleton of the backend. +Even though their modular design supports pluggability, removing those modules does not serve any purpose. For optional modules, some of them have +a provider implementation called `none`, meaning that it only provides a shell with no actual logic, typically such as telemetry. +Setting `-` to the `selector` means that this whole module will be excluded at runtime. + +The required modules are listed here: +1. **Core**. Provides the basic and major skeleton of all data analysis and stream dispatch. +1. **Cluster**. Manages multiple backend instances in a cluster, which could provide high throughput process capabilities. +1. **Storage**. Makes the analysis result persistent. +1. **Receiver Zipkin**. Provide the basic configuration for all data collection protocols. ## UI Zipkin has a web UI, automatically included in the exec jar, and is hosted by default on port 9411. When the UI loads, it reads default configuration from the `/config.json` endpoint. -Attribute | Property | Description +Attribute | Environment | Description --- | --- | --- -environment | zipkin.ui.environment | The value here becomes a label in the top-right corner. Not required. -defaultLookback | zipkin.ui.default-lookback | Default duration in millis to look back when finding traces. Affects the "Start time" element in the UI. Defaults to 900000 (15 minutes in millis). -searchEnabled | zipkin.ui.search-enabled | If the Discover screen is enabled. Defaults to true. -queryLimit | zipkin.ui.query-limit | Default limit for Find Traces. Defaults to 10. -instrumented | zipkin.ui.instrumented | Which sites this Zipkin UI covers. Regex syntax. e.g. `http:\/\/example.com\/.*` Defaults to match all websites (`.*`). -logsUrl | zipkin.ui.logs-url | Logs query service url pattern. If specified, a button will appear on the trace page and will replace {traceId} in the url by the traceId. Not required. -supportUrl | zipkin.ui.support-url | A URL where a user can ask for support. If specified, a link will be placed in the side menu to this URL, for example a page to file support tickets. Not required. -archivePostUrl | zipkin.ui.archive-post-url | Url to POST the current trace in Zipkin v2 json format. e.g. 'https://longterm/api/v2/spans'. If specified, a button will appear on the trace page accordingly. Not required. -archiveUrl | zipkin.ui.archive-url | Url to a web application serving an archived trace, templated by '{traceId}'. e.g. https://longterm/zipkin/trace/{traceId}'. This is shown in a confirmation message after a trace is successfully POSTed to the `archivePostUrl`. Not required. -dependency.enabled | zipkin.ui.dependency.enabled | If the Dependencies screen is enabled. Defaults to true. -dependency.lowErrorRate | zipkin.ui.dependency.low-error-rate | The rate of error calls on a dependency link that turns it yellow. Defaults to 0.5 (50%) set to >1 to disable. -dependency.highErrorRate | zipkin.ui.dependency.high-error-rate | The rate of error calls on a dependency link that turns it red. Defaults to 0.75 (75%) set to >1 to disable. -basePath | zipkin.ui.basepath | path prefix placed into the tag in the UI HTML; useful when running behind a reverse proxy. Default "/zipkin" - -To map properties to environment variables, change them to upper-underscore case format. For -example, if using docker you can set `ZIPKIN_UI_QUERY_LIMIT=100` to affect `$.queryLimit` in `/config.json`. - -### Trace archival -Most production Zipkin clusters store traces with a limited TTL. This makes it a bit inconvenient to -share a trace, as the link to it will expire after a few days. - -The "archive a trace" feature helps with this. Launch a second zipkin server pointing to a storage with a longer -TTL than the regular one and set the archivePostUrl and archiveUrl UI configs pointing to this second server. -Once archivePostUrl is set, a new "Archive Trace" button will appear on the trace view page. +restPort | ZIPKIN_QUERY_REST_PORT | The port for the UI interface defaults to the **ZIPKIN_SERVER_PORT** environment variable's port. If the value is greater than `0`, it will initiate a separate port to serve externally. +uiDefaultLookback | ZIPKIN_QUERY_UI_DEFAULT_LOOKBACK | Default duration in millis to look back when finding traces. Affects the "Start time" element in the UI. Defaults to 900000 (15 minutes in millis). +uiQueryLimit | ZIPKIN_QUERY_UI_QUERY_LIMIT | Default limit for Find Traces. Defaults to 10. +dependencyEnabled | ZIPKIN_QUERY_DEPENDENCY_ENABLED | If the Dependencies screen is enabled. Defaults to true. +dependencyLowErrorRate | ZIPKIN_QUERY_DEPENDENCY_LOW_ERROR_RATE | The rate of error calls on a dependency link that turns it yellow. Defaults to 0.5 (50%) set to >1 to disable. +dependencyHighErrorRate | ZIPKIN_QUERY_DEPENDENCY_HIGH_ERROR_RATE | The rate of error calls on a dependency link that turns it red. Defaults to 0.75 (75%) set to >1 to disable. +uiBasePath | ZIPKIN_QUERY_UI_BASE_PATH | path prefix placed into the tag in the UI HTML; useful when running behind a reverse proxy. Default "/zipkin" ## Storage ### In-Memory Storage -Zipkin's [In-Memory Storage](../zipkin/src/main/java/zipkin2/storage/InMemoryStorage.java) holds all -data in memory, purging older data upon a span limit. It applies when `STORAGE_TYPE` is unset or -set to the value `mem`. - * `MEM_MAX_SPANS`: Oldest traces (and their spans) will be purged first when this limit is exceeded. Default 500000 +Zipkin In-memory Storage uses an embedded H2 database for storage. By default, Zipkin utilizes this database. Example usage: ```bash @@ -213,164 +223,264 @@ $ java -jar zipkin.jar Note: this storage component was primarily developed for testing and as a means to get Zipkin server up and running quickly without external dependencies. It is not viable for high work loads. That -said, if you encounter out-of-memory errors, try decreasing `MEM_MAX_SPANS` or increasing the heap -size (-Xmx). +said, if you encounter out-of-memory errors, try increasing the heap size (-Xmx). Exampled of doubling the amount of spans held in memory: ```bash -$ MEM_MAX_SPANS=1000000 java -Xmx1G -jar zipkin.jar +$ java -Xmx1G -jar zipkin.jar ``` ### Cassandra Storage -Zipkin's [Cassandra storage component](../zipkin-storage/cassandra) supports Cassandra 3.11.3+ -and applies when `STORAGE_TYPE` is set to `cassandra3`: +Zipkin's Cassandra storage component supports Cassandra 3.11.3+ +and applies when `ZIPKIN_STORAGE` is set to `cassandra3`: - * `CASSANDRA_KEYSPACE`: The keyspace to use. Defaults to "zipkin2" - * `CASSANDRA_CONTACT_POINTS`: Comma separated list of host addresses part of Cassandra cluster. You can also specify a custom port with 'host:port'. Defaults to localhost on port 9042. - * `CASSANDRA_LOCAL_DC`: Name of the datacenter that will be considered "local" for load balancing. Defaults to "datacenter1" - * `CASSANDRA_ENSURE_SCHEMA`: Ensuring cassandra has the latest schema. If enabled tries to execute scripts in the classpath prefixed with `cassandra-schema-cql3`. Defaults to true - * `CASSANDRA_USERNAME` and `CASSANDRA_PASSWORD`: Cassandra authentication. Will throw an exception on startup if authentication fails. No default - * `CASSANDRA_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false. + * `ZIPKIN_STORAGE_CASSANDRA_KEYSPACE`: The keyspace to use. Defaults to "zipkin" + * `ZIPKIN_STORAGE_CASSANDRA_CONTACT_POINTS`: Comma separated list of host addresses part of Cassandra cluster. You can also specify a custom port with 'host:port'. Defaults to localhost on port 9042. + * `ZIPKIN_STORAGE_CASSANDRA_LOCAL_DC`: Name of the datacenter that will be considered "local" for load balancing. Defaults to "datacenter1" + * `ZIPKIN_STORAGE_CASSANDRA_ENSURE_SCHEMA`: Ensuring cassandra has the latest schema. If enabled tries to execute scripts in the classpath prefixed with `cassandra-schema-cql3`. Defaults to true + * `ZIPKIN_STORAGE_CASSANDRA_USERNAME` and `ZIPKIN_STORAGE_CASSANDRA_PASSWORD`: Cassandra authentication. Will throw an exception on startup if authentication fails. No default + * `ZIPKIN_STORAGE_CASSANDRA_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false. The following are tuning parameters which may not concern all users: - * `CASSANDRA_MAX_CONNECTIONS`: Max pooled connections per datacenter-local host. Defaults to 8 - * `CASSANDRA_INDEX_CACHE_MAX`: Maximum trace index metadata entries to cache. Zero disables caching. Defaults to 100000. - * `CASSANDRA_INDEX_CACHE_TTL`: How many seconds to cache index metadata about a trace. Defaults to 60. - * `CASSANDRA_INDEX_FETCH_MULTIPLIER`: How many more index rows to fetch than the user-supplied query limit. Defaults to 3. + * `ZIPKIN_STORAGE_CASSANDRA_MAX_CONNECTIONS`: Max pooled connections per datacenter-local host. Defaults to 8 -Example usage with Cassandra with request logging (TRACE shows query values): +Example usage with Cassandra: ```bash -$ STORAGE_TYPE=cassandra3 java -jar zipkin.jar \ ---logging.level.com.datastax.oss.driver.internal.core.tracker.RequestLogger=DEBUG +$ ZIPKIN_STORAGE=cassandra3 java -jar zipkin.jar ``` ### Elasticsearch Storage -Zipkin's [Elasticsearch storage component](../zipkin-storage/elasticsearch) -supports versions 5-7.x and applies when `STORAGE_TYPE` is set to `elasticsearch` - -The following apply when `STORAGE_TYPE` is set to `elasticsearch`: - - * `ES_HOSTS`: A comma separated list of elasticsearch base urls to connect to ex. http://host:9200. - Defaults to "http://localhost:9200". - * `ES_PIPELINE`: Indicates the ingest pipeline used before spans are indexed. No default. - * `ES_TIMEOUT`: Controls the connect, read and write socket timeouts (in milliseconds) for - Elasticsearch API. Defaults to 10000 (10 seconds) - * `ES_INDEX`: The index prefix to use when generating daily index names. Defaults to zipkin. - * `ES_DATE_SEPARATOR`: The date separator to use when generating daily index names. Defaults to '-'. - * `ES_INDEX_SHARDS`: The number of shards to split the index into. Each shard and its replicas - are assigned to a machine in the cluster. Increasing the number of shards - and machines in the cluster will improve read and write performance. Number - of shards cannot be changed for existing indices, but new daily indices - will pick up changes to the setting. Defaults to 5. - * `ES_INDEX_REPLICAS`: The number of replica copies of each shard in the index. Each shard and - its replicas are assigned to a machine in the cluster. Increasing the - number of replicas and machines in the cluster will improve read - performance, but not write performance. Number of replicas can be changed - for existing indices. Defaults to 1. It is highly discouraged to set this - to 0 as it would mean a machine failure results in data loss. - * `ES_ENSURE_TEMPLATES`: Installs Zipkin index templates when missing. Setting this to false can - lead to corrupted data when index templates mismatch expectations. If - you set this to false, you choose to troubleshoot your own data or - migration problems as opposed to relying on the community for this. - Defaults to true. - * `ES_USERNAME` and `ES_PASSWORD`: Elasticsearch basic authentication, which defaults to empty string. - Use when X-Pack security (formerly Shield) is in place. - * `ES_CREDENTIALS_FILE`: The location of a file containing Elasticsearch basic authentication - credentials, as properties. The username property is - `zipkin.storage.elasticsearch.username`, password `zipkin.storage.elasticsearch.password`. - This file is reloaded periodically, using `ES_CREDENTIALS_REFRESH_INTERVAL` - as the interval. This parameter takes precedence over ES_USERNAME and - ES_PASSWORD when specified. - * `ES_CREDENTIALS_REFRESH_INTERVAL`: Credentials refresh interval in seconds, which defaults to - 1 second. This is the maximum amount of time spans will drop due to stale - credentials. Any errors reading the credentials file occur in logs at this rate. - * `ES_HTTP_LOGGING`: When set, controls the volume of HTTP logging of the Elasticsearch API. - Options are BASIC, HEADERS, BODY - * `ES_SSL_NO_VERIFY`: When true, disables the verification of server's key certificate chain. - This is not appropriate for production. Defaults to false. - * `ES_TEMPLATE_PRIORITY`: The priority value of the composable index templates. This is only applicable - for ES version 7.8 or above. Must be set, even to 0, to use composable template +Zipkin's Elasticsearch storage component supports Elasticsearch versions 6-8.x, OpenSearch 1.1.0-1.3.10 and 2.4.0-2.8.0, and applies when `ZIPKIN_STORAGE` is set to `elasticsearch`. + +The following apply when `ZIPKIN_STORAGE` is set to `elasticsearch`: + + * `ZIPKIN_STORAGE_ES_CLUSTER_NODES`: A comma separated list of elasticsearch base nodes to connect to ex. host:9200. + Defaults to "localhost:9200". + * `ZIPKIN_STORAGE_ES_HTTP_PROTOCOL`: The protocol used when connecting to an Elasticsearch cluster. Defaults to "http". + * `ZIPKIN_NAMESPACE`: All namespaces of Index are distinguished from other indexes by prefixes. Defaults to "zipkin". + * `ZIPKIN_STORAGE_ES_CONNECT_TIMEOUT`: Connect timeout of ElasticSearch client. Defaults to 3000 (3 seconds) + * `ZIPKIN_STORAGE_ES_SOCKET_TIMEOUT`: Socket timeout of ElasticSearch client. Defaults to 30000 (30 seconds) + * `ZIPKIN_STORAGE_ES_RESPONSE_TIMEOUT`: the response timeout of ElasticSearch client (Armeria under the hood), set to 0 to disable response. Defaults to 15000 (15 seconds) + * `ZIPKIN_STORAGE_ES_NUM_HTTP_CLIENT_THREAD`: The number of threads for the underlying HTTP client to perform socket I/O. + If the value is <= 0, the number of available processors will be used. Defaults to 0. + * `ZIPKIN_ES_USER` and `ZIPKIN_ES_PASSWORD`: Elasticsearch basic authentication, which defaults to empty string. + Use when X-Pack security (formerly Shield) is in place. + * `ZIPKIN_STORAGE_ES_SSL_JKS_PATH`: The path to the Java keystore file containing an Elasticsearch SSL certificate for use with HTTPS. Defaults to "". + * `ZIPKIN_STORAGE_ES_SSL_JKS_PASS`: The password for the Java keystore file containing an Elasticsearch SSL certificate for use with HTTPS. Defaults to "". + * `ZIPKIN_ES_SECRETS_MANAGEMENT_FILE`: Secrets management file in the properties format includes the username, password, which are managed by 3rd party tool. Defaults to "". + * `ZIPKIN_STORAGE_DAY_STEP`: Represent the number of days in the one minute/hour/day index. Defaults to 1. + * `ZIPKIN_STORAGE_ES_INDEX_SHARDS_NUMBER`: The number of shards to split the index into. Each shard and its replicas + are assigned to a machine in the cluster. Increasing the number of shards + and machines in the cluster will improve read and write performance. Number + of shards cannot be changed for existing indices, but new daily indices + will pick up changes to the setting. Defaults to 1. + * `ZIPKIN_STORAGE_ES_INDEX_REPLICAS_NUMBER`: The number of replica copies of each shard in the index. Each shard and + its replicas are assigned to a machine in the cluster. Increasing the + number of replicas and machines in the cluster will improve read + performance, but not write performance. Number of replicas can be changed + for existing indices. Defaults to 1. It is highly discouraged to set this + to 0 as it would mean a machine failure results in data loss. Defaults to 1. + * `ZIPKIN_STORAGE_ES_SPECIFIC_INDEX_SETTINGS`: Specify the settings for specify index individually. + If configured, this setting has the highest priority and overrides the generic settings. + * `ZIPKIN_STORAGE_ES_SUPER_DATASET_DAY_STEP`: Represent the number of days in the zipkin span record index, + the default value is the same as dayStep when the value is less than 0 + * `ZIPKIN_STORAGE_ES_SUPER_DATASET_INDEX_SHARDS_FACTOR`: This factor provides more shards for the zipkin span record, + shards number = indexShardsNumber * superDatasetIndexShardsFactor. + Defaults to 5. + * `ZIPKIN_STORAGE_ES_SUPER_DATASET_INDEX_REPLICAS_NUMBER`: Represent the replicas number in the zipkin span record index, Defaults to 0. + * `ZIPKIN_STORAGE_ES_INDEX_TEMPLATE_ORDER`: The order of the index template. Defaults to 0. + * `ZIPKIN_STORAGE_ES_BULK_ACTIONS`: The number of requests to send to Elasticsearch in a single bulk request. Defaults to 5000. + * `ZIPKIN_STORAGE_ES_BATCH_OF_BYTES`: The number of bytes to send to Elasticsearch in a single bulk request. Defaults to 10485760 (10MB). + * `ZIPKIN_STORAGE_ES_FLUSH_INTERVAL`: The number of second to wait between bulk requests to Elasticsearch. Defaults to 5. + * `ZIPKIN_STORAGE_ES_CONCURRENT_REQUESTS`: The number of concurrent requests to Elasticsearch. Defaults to 2. + * `ZIPKIN_STORAGE_ES_LOGIC_SHARDING`: Enable shard metrics and records indices into multi-physical indices, + one index template per metric/meter aggregation function or record. Defaults to false. + * `ZIPKIN_STORAGE_ES_ENABLE_CUSTOM_ROUTING`: Custom routing can reduce the impact of searches. Instead of having to fan out + a search request to all the shards in an index, the request can be sent to + just the shard that matches the specific routing value (or values). Defaults to false. Example usage: To connect normally: ```bash -$ STORAGE_TYPE=elasticsearch ES_HOSTS=http://myhost:9200 java -jar zipkin.jar +$ ZIPKIN_STORAGE=elasticsearch ZIPKIN_STORAGE_ES_CLUSTER_NODES=myhost:9200 java -jar zipkin.jar ``` -To log Elasticsearch API requests: -```bash -$ STORAGE_TYPE=elasticsearch ES_HTTP_LOGGING=BASIC java -jar zipkin.jar +#### ElasticSearch With Https SSL Encrypting communications. + +Example: + +```yaml +storage: + selector: ${ZIPKIN_STORAGE:elasticsearch} + elasticsearch: + namespace: ${ZIPKIN_NAMESPACE:"zipkin"} + user: ${ZIPKIN_ES_USER:""} # User needs to be set when Http Basic authentication is enabled + password: ${ZIPKIN_ES_PASSWORD:""} # Password to be set when Http Basic authentication is enabled + clusterNodes: ${ZIPKIN_STORAGE_ES_CLUSTER_NODES:localhost:443} + trustStorePath: ${ZIPKIN_STORAGE_ES_SSL_JKS_PATH:"../es_keystore.jks"} + trustStorePass: ${ZIPKIN_STORAGE_ES_SSL_JKS_PASS:""} + protocol: ${SW_STORAGE_ES_HTTP_PROTOCOL:"https"} + ... +``` +- File at `trustStorePath` is being monitored. Once it is changed, the ElasticSearch client will reconnect. +- `trustStorePass` could be changed in the runtime through [**Secrets Management File Of ElasticSearch Authentication**](#secrets-management-file-of-elasticsearch-authentication). + +#### Secrets Management File Of ElasticSearch Authentication +The value of `secretsManagementFile` should point to the secrets management file absolute path. +The file includes the username, password, and JKS password of the ElasticSearch server in the properties format. +```properties +user=xxx +password=yyy +trustStorePass=zzz ``` -#### Using a custom Key Store or Trust Store (SSL) -If your Elasticsearch endpoint customized SSL configuration (for example self-signed) certificates, -you can use any of the following [subset of JSSE properties](https://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html#T6) to connect. +The major difference between using `user, password, trustStorePass` configs in the `application.yaml` file is that the **Secrets Management File** is being watched by the Zipkin server. +Once it is changed manually or through a 3rd party tool, such as [Vault](https://github.com/hashicorp/vault), +the storage provider will use the new username, password, and JKS password to establish the connection and close the old one. If the information exists in the file, +the `user/password` will be overridden. - * javax.net.ssl.keyStore - * javax.net.ssl.keyStorePassword - * javax.net.ssl.keyStoreType - * javax.net.ssl.trustStore - * javax.net.ssl.trustStorePassword - * javax.net.ssl.trustStoreType +#### Daily Index Step +Daily index step(`storage/elasticsearch/dayStep`, default 1) represents the index creation period. In this period, metrics for several days (dayStep value) are saved. -Usage example: -```bash -$ JAVA_OPTS='-Djavax.net.ssl.keyStore=keystore.p12 -Djavax.net.ssl.keyStorePassword=keypassword -Djavax.net.ssl.keyStoreType=PKCS12 -Djavax.net.ssl.trustStore=truststore.p12 -Djavax.net.ssl.trustStorePassword=trustpassword -Djavax.net.ssl.trustStoreType=PKCS12' -$ STORAGE_TYPE=elasticsearch java $JAVA_OPTS -jar zipkin.jar +In most cases, users don't need to change the value manually, as SkyWalking is designed to observe large-scale distributed systems. +But in some cases, users may want to set a long TTL value, such as more than 60 days. However, their ElasticSearch cluster may not be powerful enough due to low traffic in the production environment. +This value could be increased to 5 (or more) if users could ensure a single index could support the metrics and traces for these days (5 in this case). + +For example, if dayStep == 11, +1. Data in [2000-01-01, 2000-01-11] will be merged into the index-20000101. +1. Data in [2000-01-12, 2000-01-22] will be merged into the index-20000112. + +`storage/elasticsearch/superDatasetDayStep` overrides the `storage/elasticsearch/dayStep` if the value is positive. This would affect the zipkin span entity. + +NOTE: TTL deletion would be affected by these steps. You should set an extra dayStep in your TTL. For example, if you want to have TTL == 30 days and dayStep == 10, you are recommended to set TTL = 40. + +#### Index Settings +The following settings control the number of shards and replicas for new and existing index templates. The update only got applied after OAP reboots. +```yaml +storage: + elasticsearch: + # ...... + indexShardsNumber: ${ZIPKIN_STORAGE_ES_INDEX_SHARDS_NUMBER:1} + indexReplicasNumber: ${ZIPKIN_STORAGE_ES_INDEX_REPLICAS_NUMBER:1} + specificIndexSettings: ${ZIPKIN_STORAGE_ES_SPECIFIC_INDEX_SETTINGS:""} + superDatasetIndexShardsFactor: ${ZIPKIN_STORAGE_ES_SUPER_DATASET_INDEX_SHARDS_FACTOR:5} + superDatasetIndexReplicasNumber: ${ZIPKIN_STORAGE_ES_SUPER_DATASET_INDEX_REPLICAS_NUMBER:0} ``` +The following table shows the relationship between those config items and Elasticsearch `index number_of_shards/number_of_replicas`. +And also you can [specify the settings for each index individually.](#specify-settings-for-each-elasticsearch-index-individually) -Under the scenes, these map to properties prefixed `zipkin.storage.elasticsearch.ssl.`, which affect -the Armeria client used to connect to Elasticsearch. +| index | number_of_shards | number_of_replicas | description | +|--------------------------------------|------------------|----------------------|-------------| +| zipkin_metrics-all-`${day-format}` | indexShardsNumber | indexReplicasNumber | All metrics/meters generated by zipkin spans, and metadata of service | +| zipkin_zipkin_span-`${day-format}` | indexShardsNumber * superDatasetIndexShardsFactor | superDatasetIndexReplicasNumber | Zipkin trace spans | -The above properties allow the most common SSL setup to work out of box. If you need more -customization, please make a comment in [this issue](https://github.com/openzipkin/zipkin/issues/2774). +##### Advanced Configurations For Elasticsearch Index +You can add advanced configurations in `JSON` format to set `ElasticSearch index settings` by following [ElasticSearch doc](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html) -#### Automatic Index Creation -Zipkin will automatically create new indices as needed. Elasticsearch by default [allows](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-index_.html#index-creation) automatic creation of said indices, though your local install may have been configured to disallow it. You can verify this in the cluster settings: `action.auto_create_index: false`. +For example, set [translog](https://www.elastic.co/guide/en/elasticsearch/reference/master/index-modules-translog.html) settings: -### Legacy (v1) storage components -The following components are no longer encouraged, but exist to help aid -transition to supported ones. These are indicated as "v1" as they use -data layouts based on Zipkin's V1 Thrift model, as opposed to the -simpler v2 data model currently used. +```yaml +storage: + elasticsearch: + # ...... + advanced: ${ZIPKIN_STORAGE_ES_ADVANCED:"{\"index.translog.durability\":\"request\",\"index.translog.sync_interval\":\"5s\"}"} +``` -#### MySQL Storage -Zipkin's [MySQL component](../zipkin-storage/mysql-v1) is tested against MySQL -5.7 and applies when `STORAGE_TYPE` is set to `mysql`: +##### Specify Settings For Each Elasticsearch Index Individually +You can specify the settings for one or more indexes individually by using `SW_STORAGE_ES_SPECIFIC_INDEX_SETTINGS`. + +**NOTE:** +Supported settings: +- number_of_shards +- number_of_replicas + +**NOTE:** These settings have the highest priority and will override the existing +generic settings mentioned in [index settings doc](#index-settings). + +The settings are in `JSON` format. The index name here is logic entity name, which should exclude the `${ZIPKIN_NAMESPACE}` which is `zipkin` by default, e.g. +```json +{ + "metrics-all":{ + "number_of_shards":"3", + "number_of_replicas":"2" + }, + "segment":{ + "number_of_shards":"6", + "number_of_replicas":"1" + } +} +``` - * `MYSQL_DB`: The database to use. Defaults to "zipkin". - * `MYSQL_USER` and `MYSQL_PASS`: MySQL authentication, which defaults to empty string. - * `MYSQL_HOST`: Defaults to localhost - * `MYSQL_TCP_PORT`: Defaults to 3306 - * `MYSQL_MAX_CONNECTIONS`: Maximum concurrent connections, defaults to 10 - * `MYSQL_USE_SSL`: Requires `javax.net.ssl.trustStore` and `javax.net.ssl.trustStorePassword`, defaults to false. +This configuration in the YAML file is like this, +```yaml +storage: + elasticsearch: + # ...... + specificIndexSettings: ${ZIPKIN_STORAGE_ES_SPECIFIC_INDEX_SETTINGS:"{\"metrics-all\":{\"number_of_shards\":\"3\",\"number_of_replicas\":\"2\"},\"segment\":{\"number_of_shards\":\"6\",\"number_of_replicas\":\"1\"}}"} +``` + +#### Recommended ElasticSearch server-side configurations +You could add the following configuration to `elasticsearch.yml`, and set the value based on your environment. + +```yml +# In tracing scenario, consider to set more than this at least. +thread_pool.index.queue_size: 1000 # Only suitable for ElasticSearch 6 +thread_pool.write.queue_size: 1000 # Suitable for ElasticSearch 6 and 7 + +# When you face a query error on the traces page, remember to check this. +index.max_result_window: 1000000 +``` + +We strongly recommend that you read more about these configurations from ElasticSearch's official documentation since they directly impact the performance of ElasticSearch. -Note: This module is not recommended for production usage. Before using this, -you must [apply the schema](../zipkin-storage/mysql-v1#applying-the-schema). +### BanyanDB storage components -Alternatively you can use `MYSQL_JDBC_URL` and specify the complete JDBC url yourself. Note that the URL constructed by -using the separate settings above will also include the following parameters: -`?autoReconnect=true&useSSL=false&useUnicode=yes&characterEncoding=UTF-8`. If you specify the JDBC url yourself, add -these parameters as well. +[BanyanDB](https://github.com/apache/skywalking-banyandb) is a dedicated storage implementation developed by the SkyWalking Team and the community. +Currently, BanyanDB is still in the PoC stage and it is not recommended to use it in a production environment. + +The following apply when `ZIPKIN_STORAGE` is set to `banyandb`: + + * `SW_STORAGE_BANYANDB_TARGETS`: A comma-separated list of BanyanDB targets. ex, `localhost:17912,localhost:17913`. Defaults to `localhost:17912`. + * `ZIPKIN_STORAGE_BANYANDB_MAX_BULK_SIZE`: The max bulk size of BanyanDB. Defaults to `5000`. + * `ZIPKIN_STORAGE_BANYANDB_FLUSH_INTERVAL`: The flush interval of BanyanDB. Defaults to `15`. + * `ZIPKIN_STORAGE_BANYANDB_METRICS_SHARDS_NUMBER`: The number of shards of metrics index. Defaults to `1`. + * `ZIPKIN_STORAGE_BANYANDB_SUPERDATASET_SHARDS_FACTOR`: The shards factor of zipkin span record index. Defaults to `2`. + * `ZIPKIN_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS`: The number of concurrent write threads. Defaults to `15`. + * `ZIPKIN_STORAGE_BANYANDB_BLOCK_INTERVAL_HOURS`: The block interval hours of BanyanDB. Defaults to `24` hour. + * `ZIPKIN_STORAGE_BANYANDB_SEGMENT_INTERVAL_DAYS`: The segment interval days of BanyanDB. Defaults to `1` days. + * `ZIPKIN_STORAGE_BANYANDB_SUPER_DATASET_BLOCK_INTERVAL_HOURS`: The zipkin span record block interval hours of BanyanDB. Defaults to `4` hour. + * `ZIPKIN_STORAGE_BANYANDB_SUPER_DATASET_SEGMENT_INTERVAL_DAYS`: The zipkin span record segment interval days of BanyanDB. Defaults to `1` days. + * `ZIPKIN_STORAGE_BANYANDB_SPECIFIC_GROUP_SETTINGS`: The specific group settings of BanyanDB. ex, `{"group1": {"blockIntervalHours": 4, "segmentIntervalDays": 1}}`. Defaults to "". Example usage: ```bash -$ STORAGE_TYPE=mysql MYSQL_USER=root java -jar zipkin.jar +$ ZIPKIN_STORAGE=banyandb java -jar zipkin.jar ``` -### Throttled Storage (Experimental) -These settings can be used to help tune the rate at which Zipkin flushes data to another, underlying -`StorageComponent` (such as Elasticsearch): +### MySQL/PostgreSQL Storage +Zipkin's MySQL/PostgreSQL component is tested against MySQL 5.7, PostgreSQL 9 and applies when `ZIPKIN_STORAGE` is set to `mysql`/`postgresql`: + + * `ZIPKIN_JDBC_URL`: The connection string to MySQL, ex. `jdbc:mysql://host/dbname`. + Defaults to `jdbc:mysql://localhost:3306/zipkin?rewriteBatchedStatements=true&allowMultiQueries=true` when using mysql + and `jdbc:postgresql://localhost:5432/zipkin` when using postgresql. + * `ZIPKIN_DATA_SOURCE_USER` and `ZIPKIN_DATA_SOURCE_PASSWORD`: MySQL authentication, which defaults to `zipkin` and `zipkin`. + * `ZIPKIN_DATA_SOURCE_CACHE_PREP_STMTS`: Whether prepared statements should be cached or not. Defaults to `true`. + * `ZIPKIN_DATA_SOURCE_PREP_STMT_CACHE_SQL_SIZE`: The number of prepared statements that the driver will cache per connection. Defaults to 250. + * `ZIPKIN_DATA_SOURCE_PREP_STMT_CACHE_SQL_LIMIT`: The number of queries that can be cached in an LRU cache, to limit the memory cost of caching prepared statements. Defaults to 2048. + * `ZIPKIN_DATA_SOURCE_USE_SERVER_PREP_STMTS`: Whether the driver should use a per-connection cache of prepared statements. Defaults to `true`. + * `ZIPKIN_STORAGE_MAX_SIZE_OF_BATCH_SQL`: Maximum number of statements to batch in one go. Defaults to 2000. + * `ZIPKIN_STORAGE_ASYNC_BATCH_PERSISTENT_POOL_SIZE`: Maximum number of threads writing to MySQL. Defaults to 4. - * `STORAGE_THROTTLE_ENABLED`: Enables throttling - * `STORAGE_THROTTLE_MIN_CONCURRENCY`: Minimum number of Threads to use for writing to storage. - * `STORAGE_THROTTLE_MAX_CONCURRENCY`: Maximum number of Threads to use for writing to storage. - * `STORAGE_THROTTLE_MAX_QUEUE_SIZE`: How many messages to buffer while all Threads are writing data before abandoning a message (0 = no buffering). +Note: This module is not recommended for production usage. -As this feature is experimental, it is not recommended to run this in production environments. +Example usage: + +```bash +$ ZIPKIN_STORAGE=mysql ZIPKIN_DATA_SOURCE_USER=zipkin ZIPKIN_DATA_SOURCE_PASSWORD=zipkin java -jar zipkin.jar +``` ## Collector @@ -378,58 +488,58 @@ As this feature is experimental, it is not recommended to run this in production The HTTP collector is enabled by default. It accepts spans via `POST /api/v1/spans` and `POST /api/v2/spans`. The HTTP collector supports the following configuration: -Property | Environment Variable | Description +Environment Variable | Default | Description --- | --- | --- -`zipkin.collector.http.enabled` | `COLLECTOR_HTTP_ENABLED` | `false` disables the HTTP collector. Defaults to `true`. +`ZIPKIN_RECEIVER_ZIPKIN_HTTP` | `default` | `-` disables the HTTP collector. ### Scribe (Legacy) Collector -A collector supporting Scribe is enabled when `COLLECTOR_SCRIBE_ENABLED=true`. New +A collector supporting Scribe is enabled when `ZIPKIN_RECEIVER_ZIPKIN_SCRIBE=default`. New sites are discouraged from using this collector as Scribe is an archived technology. -Environment Variable | Property | Description +Environment Variable | Default | Description --- | --- | --- -`COLLECTOR_PORT` | `zipkin.collector.scribe.port` | The port to listen for thrift RPC scribe requests. Defaults to 9410 -`SCRIBE_CATEGORY` | `zipkin.collector.scribe.category` | Category zipkin spans will be consumed from. Defaults to `zipkin` +`ZIPKIN_COLLECTOR_PORT` | `9410` | The port to listen for thrift RPC scribe requests. +`ZIPKIN_SCRIBE_CATEGORY` | `zipkin` | Category zipkin spans will be consumed from. ### ActiveMQ Collector -The [ActiveMQ Collector](../zipkin-collector/activemq) is enabled when `ACTIVEMQ_URL` is set to a v5.x broker. The following settings apply in this case. +The ActiveMQ Collector is enabled when `ZIPKIN_RECEIVER_ZIPKIN_ACTIVEMQ` is set to `default`. The following settings apply in this case. -Environment Variable | Property | Description +Environment Variable | Default | Description --- | --- | --- -`COLLECTOR_ACTIVEMQ_ENABLED` | `zipkin.collector.activemq.enabled` | `false` disables the ActiveMQ collector. Defaults to `true`. -`ACTIVEMQ_URL` | `zipkin.collector.activemq.url` | [Connection URL](https://activemq.apache.org/uri-protocols) to the ActiveMQ broker, ex. `tcp://localhost:61616` or `failover:(tcp://localhost:61616,tcp://remotehost:61616)` -`ACTIVEMQ_QUEUE` | `zipkin.collector.activemq.queue` | Queue from which to collect span messages. Defaults to `zipkin` -`ACTIVEMQ_CLIENT_ID_PREFIX` | `zipkin.collector.activemq.client-id-prefix` | Client ID prefix for queue consumers. Defaults to `zipkin` -`ACTIVEMQ_CONCURRENCY` | `zipkin.collector.activemq.concurrency` | Number of concurrent span consumers. Defaults to `1` -`ACTIVEMQ_USERNAME` | `zipkin.collector.activemq.username` | Optional username to connect to the broker -`ACTIVEMQ_PASSWORD`| `zipkin.collector.activemq.password` | Optional password to connect to the broker +`ZIPKIN_RECEIVER_ZIPKIN_ACTIVEMQ` | `-` | `default` enable the ActiveMQ collector. +`ZIPKIN_ACTIVEMQ_URL` | `` | [Connection URL](https://activemq.apache.org/uri-protocols) to the ActiveMQ broker, ex. `tcp://localhost:61616` or `failover:(tcp://localhost:61616,tcp://remotehost:61616)` +`ZIPKIN_ACTIVEMQ_QUEUE` | `zipkin` | Queue from which to collect span messages. +`ZIPKIN_ACTIVEMQ_CLIENT_ID_PREFIX` | `zipkin` | Client ID prefix for queue consumers. Defaults to `zipkin` +`ZIPKIN_ACTIVEMQ_CONCURRENCY` | `1` | Number of concurrent span consumers. +`ZIPKIN_ACTIVEMQ_USERNAME` | `` | Optional username to connect to the broker +`ZIPKIN_ACTIVEMQ_PASSWORD`| `` | Optional password to connect to the broker Example usage: ```bash -$ ACTIVEMQ_URL=tcp://localhost:61616 java -jar zipkin.jar +$ ZIPKIN_RECEIVER_ZIPKIN_ACTIVEMQ=default ZIPKIN_ACTIVEMQ_URL=tcp://localhost:61616 java -jar zipkin.jar ``` ### Kafka Collector -The Kafka collector is enabled when `KAFKA_BOOTSTRAP_SERVERS` is set to -a v0.10+ server. The following settings apply in this case. Some settings -correspond to "New Consumer Configs" in [Kafka documentation](https://kafka.apache.org/documentation/#newconsumerconfigs). +The Kafka collector is enabled when `ZIPKIN_RECEIVER_ZIPKIN_KAFKA` is set to `default`. -Variable | New Consumer Config | Description +Environment Variable | Default | Description --- | --- | --- -`COLLECTOR_KAFKA_ENABLED` | N/A | `false` disables the Kafka collector. Defaults to `true`. -`KAFKA_BOOTSTRAP_SERVERS` | bootstrap.servers | Comma-separated list of brokers, ex. 127.0.0.1:9092. No default -`KAFKA_GROUP_ID` | group.id | The consumer group this process is consuming on behalf of. Defaults to `zipkin` -`KAFKA_TOPIC` | N/A | Comma-separated list of topics that zipkin spans will be consumed from. Defaults to `zipkin` -`KAFKA_STREAMS` | N/A | Count of threads consuming the topic. Defaults to `1` +`ZIPKIN_RECEIVER_ZIPKIN_KAFKA` | `-` | `default` enable the Kafka collector. +`ZIPKIN_KAFKA_SERVERS` | `localhost:9092` | Comma-separated list of brokers, ex. `127.0.0.1:9092`. +`ZIPKIN_KAFKA_GROUP_ID` | `zipkin` | The consumer group this process is consuming on behalf of. +`ZIPKIN_KAFKA_TOPIC` | `zipkin` | Comma-separated list of topics that zipkin spans will be consumed from. +`ZIPKIN_KAFKA_CONSUMER_CONFIG` | `{\"auto.offset.reset\":\"earliest\",\"enable.auto.commit\":true}` | Kafka consumer config, JSON format as Properties. If it contains the same key with above, would override. +`ZIPKIN_KAFKA_CONSUMERS` | `1` | Number of consumers reading from the topic. +`ZIPKIN_KAFKA_HANDLER_THREAD_POOL_SIZE` | `-1` | The size of the thread pool that the Kafka consumer would use to schedule data processing. If <= 0, the default value is the number of processors available to the Java virtual machine. +`ZIPKIN_KAFKA_HANDLER_THREAD_POOL_QUEUE_SIZE` | `-1` | The size of the queue that the Kafka consumer would use to buffer data to be processed. Example usage: ```bash -$ KAFKA_BOOTSTRAP_SERVERS=127.0.0.1:9092 \ - java -jar zipkin.jar +$ ZIPKIN_RECEIVER_ZIPKIN_KAFKA=default ZIPKIN_KAFKA_SERVERS=127.0.0.1:9092 java -jar zipkin.jar ``` #### Other Kafka consumer properties @@ -439,12 +549,13 @@ addition to the ones with explicit properties defined by the collector. In this prefix that property name with `zipkin.collector.kafka.overrides` and pass it as a system property argument. -For example, to override `auto.offset.reset`, you can set a system property named -`zipkin.collector.kafka.overrides.auto.offset.reset`: +For example, to override `auto.offset.reset`, you can set environment variable +`ZIPKIN_KAFKA_CONSUMER_CONFIG={"auto.offset.reset":"earliest"}`: ```bash -$ KAFKA_BOOTSTRAP_SERVERS=127.0.0.1:9092 \ - java -Dzipkin.collector.kafka.overrides.auto.offset.reset=latest -jar zipkin.jar +$ ZIPKIN_RECEIVER_ZIPKIN_KAFKA=default ZIPKIN_KAFKA_SERVERS=127.0.0.1:9092 \ + ZIPKIN_KAFKA_CONSUMER_CONFIG={"auto.offset.reset":"latest"} \ + java -jar zipkin.jar ``` #### Detailed examples @@ -452,73 +563,65 @@ $ KAFKA_BOOTSTRAP_SERVERS=127.0.0.1:9092 \ Example targeting Kafka running in Docker: ```bash -$ export KAFKA_BOOTSTRAP_SERVERS=$(docker-machine ip `docker-machine active`) +$ export ZIPKIN_KAFKA_SERVERS=$(docker-machine ip `docker-machine active`) # Run Kafka in the background $ docker run -d -p 9092:9092 \ - --env ADVERTISED_HOST=$KAFKA_BOOTSTRAP_SERVERS \ + --env ADVERTISED_HOST=ZIPKIN_KAFKA_SERVERS \ --env AUTO_CREATE_TOPICS=true \ spotify/kafka -# Start the zipkin server, which reads $KAFKA_BOOTSTRAP_SERVERS -$ java -jar zipkin.jar +# Start the zipkin server, which reads $ZIPKIN_KAFKA_SERVERS +$ ZIPKIN_RECEIVER_ZIPKIN_KAFKA=default java -jar zipkin.jar ``` Multiple bootstrap servers: ```bash -$ KAFKA_BOOTSTRAP_SERVERS=broker1.local:9092,broker2.local:9092 \ +$ ZIPKIN_RECEIVER_ZIPKIN_KAFKA=default ZIPKIN_KAFKA_SERVERS=broker1.local:9092,broker2.local:9092 \ java -jar zipkin.jar ``` Alternate topic name(s): ```bash -$ KAFKA_BOOTSTRAP_SERVERS=127.0.0.1:9092 \ - java -Dzipkin.collector.kafka.topic=zapkin,zipken -jar zipkin.jar -``` - -Specifying bootstrap servers as a system property, instead of an environment variable: - -```bash -$ java -Dzipkin.collector.kafka.bootstrap-servers=127.0.0.1:9092 \ - -jar zipkin.jar +$ ZIPKIN_RECEIVER_ZIPKIN_KAFKA=default ZIPKIN_KAFKA_SERVERS=127.0.0.1:9092 ZIPKIN_KAFKA_TOPIC=zapkin,zipken \ + java -jar zipkin.jar ``` ### RabbitMQ collector -The [RabbitMQ collector](../zipkin-collector/rabbitmq) will be enabled when the `addresses` or `uri` for the RabbitMQ server(s) is set. +The RabbitMQ collector will be enabled when the `ZIPKIN_RECEIVER_ZIPKIN_RABBITMQ` is set to `default`. + +Environment Variable | Default | Description +--- | --- | --- +`ZIPKIN_RECEIVER_ZIPKIN_RABBITMQ` | `-` | `default` enable the RabbitMQ collector. +`ZIPKIN_RECEIVER_RABBIT_ADDRESSES` | `` | Comma-separated list of addresses to which the client will connect. +`ZIPKIN_RECEIVER_RABBIT_CONCURRENCY` | `1` | Number of concurrent consumers. +`ZIPKIN_RECEIVER_RABBIT_CONNECTION_TIMEOUT` | `60000` | TCP connection timeout in milliseconds. +`ZIPKIN_RECEIVER_RABBIT_USER` | `guest` | Username to use when authenticating to the server. +`ZIPKIN_RECEIVER_RABBIT_PASSWORD` | `guest` | Password to use when authenticating to the server. +`ZIPKIN_RECEIVER_RABBIT_QUEUE` | `zipkin` | Name of the queue to listen for spans. +`ZIPKIN_RECEIVER_RABBIT_VIRTUAL_HOST` | `/` | Virtual host to use when connecting to the RabbitMQ. +`ZIPKIN_RECEIVER_RABBIT_USE_SSL` | `false` | Whether to use SSL when connecting. +`ZIPKIN_RECEIVER_RABBIT_URI` | `` | The RabbitMQ URI to connect to. When set, it overrides all other RabbitMQ connection properties. Example usage: ```bash -$ RABBIT_ADDRESSES=localhost java -jar zipkin.jar +$ ZIPKIN_RECEIVER_ZIPKIN_RABBITMQ=default ZIPKIN_RECEIVER_RABBIT_ADDRESSES=127.0.0.1:5672 java -jar zipkin.jar ``` ### gRPC Collector (Experimental) -You can enable a gRPC span collector endpoint by setting `COLLECTOR_GRPC_ENABLED=true`. The -`zipkin.proto3.SpanService/Report` endpoint will run on the same port as normal HTTP (9411). +You can enable a gRPC span collector endpoint by setting `ZIPKIN_RECEIVER_ZIPKIN_GRPC=default`. The +`zipkin.proto3.SpanService/Report` [endpoint](https://github.com/openzipkin/zipkin-api/blob/7692ca7be4dc3be9225db550d60c4d30e6e9ec59/zipkin.proto#L232) will run on the same port as normal HTTP (9411). Example usage: ```bash -$ COLLECTOR_GRPC_ENABLED=true java -jar zipkin.jar +$ ZIPKIN_RECEIVER_ZIPKIN_GRPC=true java -jar zipkin.jar ``` As this service is experimental, it is not recommended to run this in production environments. -## Self-Tracing -Self tracing exists to help troubleshoot performance of the zipkin-server. Production deployments -who enable self-tracing should lower the sample rate from 1.0 (100%) to a much smaller rate, like -0.001 (0.1% or 1 out of 1000). - -When `zipkin.self-tracing.enabled=true`, Zipkin will self-trace calls to the API under the service -name "zipkin-server". - -Variable | Property | Description ---- | --- | --- -SELF_TRACING_ENABLED | zipkin.self-tracing.enabled | Set to true to enable self-tracing. Defaults to false -SELF_TRACING_SAMPLE_RATE | zipkin.self-tracing.sample-rate | Percentage of self-traces to retain, defaults to always sample (1.0). -SELF_TRACING_FLUSH_INTERVAL | zipkin.self-tracing.flush-interval | Interval in seconds to flush self-tracing data to storage. Defaults to 1 - ### 128-bit trace IDs Zipkin supports 64 and 128-bit trace identifiers, typically serialized @@ -579,23 +682,27 @@ Zipkin-server can be made to run with TLS if needed: ```bash # assuming you generate the key like this -keytool -genkeypair -alias mysite -keyalg RSA -keysize 2048 -storetype PKCS12 -keystore zipkin.p12 -validity 3650 +openssl genpkey -algorithm RSA -out private-key.pem +openssl req -new -key private-key.pem -out certificate-request.csr +openssl x509 -req -in certificate-request.csr -signkey private-key.pem -out certificate.pem -java -jar zipkin.jar --armeria.ssl.key-store=zipkin.p12 --armeria.ssl.key-store-type=PKCS12 --armeria.ssl.key-store-password=123123 --armeria.ssl.key-alias=mysite --armeria.ssl.enabled=true --armeria.ports[0].port=9411 --armeria.ports[0].protocols[0]=https +ZIPKIN_SERVER_SSL_ENABLED=true ZIPKIN_SERVER_SSL_KEY_PATH=./private-key.pem ZIPKIN_SERVER_SSL_CERT_CHAIN_PATH=./certificate.pem java -jar zipkin.jar ``` ## Running with Docker Released versions of zipkin-server are published to Docker Hub as `openzipkin/zipkin`. -See [docker-zipkin](https://github.com/openzipkin/docker-zipkin) for details. +See [docker](./../docker) for details. ## Building locally To build and run the server from the currently checked out source, enter the following. ```bash +# Init submodules +git submodule update --init --recursive # Build the server and also make its dependencies -$ ./mvnw -T1C -q --batch-mode -DskipTests --also-make -pl zipkin-server clean package +$ ./mvnw -T1C -q --batch-mode -DskipTests -Dcheckstyle.skip=true --also-make -pl :zipkin-server clean package # Run the server -$ java -jar ./zipkin-server/target/zipkin-server-*exec.jar +$ java -jar ./zipkin-server/server-starter/target/zipkin-server-*exec.jar # or Run the slim server -$ java -jar ./zipkin-server/target/zipkin-server-*slim.jar +$ java -jar ./zipkin-server/server-starter/target/zipkin-server-*slim.jar ``` diff --git a/zipkin-server/server-starter/src/main/resources/application.yml b/zipkin-server/server-starter/src/main/resources/application.yml index 48cba1b410b..bd32c8419f2 100644 --- a/zipkin-server/server-starter/src/main/resources/application.yml +++ b/zipkin-server/server-starter/src/main/resources/application.yml @@ -38,7 +38,6 @@ core: serverIdleTimeOut: ${ZIPKIN_SERVER_IDLE_TIMEOUT:30000} serverAcceptQueueSize: ${ZIPKIN_SERVER_QUEUE_SIZE:0} serverMaxRequestHeaderSize: ${ZIPKIN_SERVER_MAX_REQUEST_HEADER_SIZE:8192} - serverThreadPoolQueueSize: ${ZIPKIN_SERVER_POOL_QUEUE_SIZE:-1} serverEnableTLS: ${ZIPKIN_SERVER_SSL_ENABLED:false} serverTLSKeyPath: ${ZIPKIN_SERVER_SSL_KEY_PATH:""} serverTLSCertChainPath: ${ZIPKIN_SERVER_SSL_CERT_CHAIN_PATH:""} @@ -76,14 +75,6 @@ storage: &storage # flush the bulk every 5 seconds whatever the number of requests flushInterval: ${ZIPKIN_STORAGE_ES_FLUSH_INTERVAL:5} concurrentRequests: ${ZIPKIN_STORAGE_ES_CONCURRENT_REQUESTS:2} # the number of concurrent requests - resultWindowMaxSize: ${ZIPKIN_STORAGE_ES_QUERY_MAX_WINDOW_SIZE:10000} - metadataQueryMaxSize: ${ZIPKIN_STORAGE_ES_QUERY_MAX_SIZE:10000} - scrollingBatchSize: ${ZIPKIN_STORAGE_ES_SCROLLING_BATCH_SIZE:5000} - segmentQueryMaxSize: ${ZIPKIN_STORAGE_ES_QUERY_SEGMENT_SIZE:200} - profileTaskQueryMaxSize: ${ZIPKIN_STORAGE_ES_QUERY_PROFILE_TASK_SIZE:200} - profileDataQueryBatchSize: ${ZIPKIN_STORAGE_ES_QUERY_PROFILE_DATA_BATCH_SIZE:100} - oapAnalyzer: ${ZIPKIN_STORAGE_ES_OAP_ANALYZER:"{\"analyzer\":{\"oap_analyzer\":{\"type\":\"stop\"}}}"} # the oap analyzer. - oapLogAnalyzer: ${ZIPKIN_STORAGE_ES_OAP_LOG_ANALYZER:"{\"analyzer\":{\"oap_log_analyzer\":{\"type\":\"standard\"}}}"} # the oap log analyzer. It could be customized by the ES analyzer configuration to support more language log formats, such as Chinese log, Japanese log and etc. advanced: ${ZIPKIN_STORAGE_ES_ADVANCED:""} # Enable shard metrics and records indices into multi-physical indices, one index template per metric/meter aggregation function or record. logicSharding: ${ZIPKIN_STORAGE_ES_LOGIC_SHARDING:false} @@ -106,37 +97,33 @@ storage: &storage dataSource.prepStmtCacheSize: ${ZIPKIN_DATA_SOURCE_PREP_STMT_CACHE_SQL_SIZE:250} dataSource.prepStmtCacheSqlLimit: ${ZIPKIN_DATA_SOURCE_PREP_STMT_CACHE_SQL_LIMIT:2048} dataSource.useServerPrepStmts: ${ZIPKIN_DATA_SOURCE_USE_SERVER_PREP_STMTS:true} - metadataQueryMaxSize: ${ZIPKIN_STORAGE_MYSQL_QUERY_MAX_SIZE:5000} maxSizeOfBatchSql: ${ZIPKIN_STORAGE_MAX_SIZE_OF_BATCH_SQL:2000} asyncBatchPersistentPoolSize: ${ZIPKIN_STORAGE_ASYNC_BATCH_PERSISTENT_POOL_SIZE:4} postgresql: properties: jdbcUrl: ${ZIPKIN_JDBC_URL:"jdbc:postgresql://localhost:5432/zipkin"} - dataSource.user: ${ZIPKIN_DATA_SOURCE_USER:postgres} - dataSource.password: ${ZIPKIN_DATA_SOURCE_PASSWORD:123456} + dataSource.user: ${ZIPKIN_DATA_SOURCE_USER:zipkin} + dataSource.password: ${ZIPKIN_DATA_SOURCE_PASSWORD:zipkin} dataSource.cachePrepStmts: ${ZIPKIN_DATA_SOURCE_CACHE_PREP_STMTS:true} dataSource.prepStmtCacheSize: ${ZIPKIN_DATA_SOURCE_PREP_STMT_CACHE_SQL_SIZE:250} dataSource.prepStmtCacheSqlLimit: ${ZIPKIN_DATA_SOURCE_PREP_STMT_CACHE_SQL_LIMIT:2048} dataSource.useServerPrepStmts: ${ZIPKIN_DATA_SOURCE_USE_SERVER_PREP_STMTS:true} - metadataQueryMaxSize: ${ZIPKIN_STORAGE_MYSQL_QUERY_MAX_SIZE:5000} maxSizeOfBatchSql: ${ZIPKIN_STORAGE_MAX_SIZE_OF_BATCH_SQL:2000} asyncBatchPersistentPoolSize: ${ZIPKIN_STORAGE_ASYNC_BATCH_PERSISTENT_POOL_SIZE:4} banyandb: - host: ${ZIPKIN_STORAGE_BANYANDB_HOST:127.0.0.1} - port: ${ZIPKIN_STORAGE_BANYANDB_PORT:17912} + targets: ${SW_STORAGE_BANYANDB_TARGETS:127.0.0.1:17912} maxBulkSize: ${ZIPKIN_STORAGE_BANYANDB_MAX_BULK_SIZE:5000} flushInterval: ${ZIPKIN_STORAGE_BANYANDB_FLUSH_INTERVAL:15} metricsShardsNumber: ${ZIPKIN_STORAGE_BANYANDB_METRICS_SHARDS_NUMBER:1} recordShardsNumber: ${ZIPKIN_STORAGE_BANYANDB_RECORD_SHARDS_NUMBER:1} superDatasetShardsFactor: ${ZIPKIN_STORAGE_BANYANDB_SUPERDATASET_SHARDS_FACTOR:2} concurrentWriteThreads: ${ZIPKIN_STORAGE_BANYANDB_CONCURRENT_WRITE_THREADS:15} - profileTaskQueryMaxSize: ${ZIPKIN_STORAGE_BANYANDB_PROFILE_TASK_QUERY_MAX_SIZE:200} # the max number of fetch task in a request blockIntervalHours: ${ZIPKIN_STORAGE_BANYANDB_BLOCK_INTERVAL_HOURS:24} # Unit is hour segmentIntervalDays: ${ZIPKIN_STORAGE_BANYANDB_SEGMENT_INTERVAL_DAYS:1} # Unit is day superDatasetBlockIntervalHours: ${ZIPKIN_STORAGE_BANYANDB_SUPER_DATASET_BLOCK_INTERVAL_HOURS:4} # Unit is hour superDatasetSegmentIntervalDays: ${ZIPKIN_STORAGE_BANYANDB_SUPER_DATASET_SEGMENT_INTERVAL_DAYS:1} # Unit is day specificGroupSettings: ${ZIPKIN_STORAGE_BANYANDB_SPECIFIC_GROUP_SETTINGS:""} # For example, {"group1": {"blockIntervalHours": 4, "segmentIntervalDays": 1}} - cassandra: + cassandra3: keyspace: ${ZIPKIN_STORAGE_CASSANDRA_KEYSPACE:zipkin} # Comma separated list of host addresses part of Cassandra cluster. Ports default to 9042 but you can also specify a custom port with 'host:port'. contactPoints: ${ZIPKIN_STORAGE_CASSANDRA_CONTACT_POINTS:localhost} @@ -150,9 +137,8 @@ storage: &storage # Max pooled connections per datacenter-local host. maxConnections: ${ZIPKIN_STORAGE_CASSANDRA_MAX_CONNECTIONS:8} # Using ssl for connection, rely on Keystore - use-ssl: ${ZIPKIN_STORAGE_CASSANDRA_USE_SSL:false} - maxSizeOfBatchCql: ${ZIPKIN_STORAGE_CASSANDRA_MAX_SIZE_OF_BATCH_CQL:2000} - asyncBatchPersistentPoolSize: ${ZIPKIN_STORAGE_CASSANDRA_ASYNC_BATCH_PERSISTENT_POOL_SIZE:4} + useSsl: ${ZIPKIN_STORAGE_CASSANDRA_USE_SSL:false} + asyncBatchPersistentPoolSize: ${ZIPKIN_STORAGE_CASSANDRA_ASYNC_BATCH_PERSISTENT_POOL_SIZE:1} zipkin-dependency-storage-ext: *storage @@ -189,6 +175,8 @@ receiver-zipkin-activemq: url: ${ZIPKIN_ACTIVEMQ_URL:} # Queue from which to collect span messages. queue: ${ZIPKIN_ACTIVEMQ_QUEUE:zipkin} + # Client ID prefix for queue consumers. + clientIdPrefix: ${ZIPKIN_ACTIVEMQ_CLIENT_ID_PREFIX:zipkin} # Number of concurrent span consumers. concurrency: ${ZIPKIN_ACTIVEMQ_CONCURRENCY:1} # Optional username to connect to the broker @@ -197,16 +185,16 @@ receiver-zipkin-activemq: password: ${ZIPKIN_ACTIVEMQ_PASSWORD:} receiver-zipkin-rabbitmq: - selector: ${ZIPKIN_RECEIVER_ZIPKIN_rabbitmq:-} + selector: ${ZIPKIN_RECEIVER_ZIPKIN_RABBITMQ:-} default: # RabbitMQ server address list (comma-separated list of host:port) addresses: ${ZIPKIN_RECEIVER_RABBIT_ADDRESSES:} concurrency: ${ZIPKIN_RECEIVER_RABBIT_CONCURRENCY:1} # TCP connection timeout in milliseconds connection-timeout: ${ZIPKIN_RECEIVER_RABBIT_CONNECTION_TIMEOUT:60000} + username: ${ZIPKIN_RECEIVER_RABBIT_USER:guest} password: ${ZIPKIN_RECEIVER_RABBIT_PASSWORD:guest} queue: ${ZIPKIN_RECEIVER_RABBIT_QUEUE:zipkin} - username: ${ZIPKIN_RECEIVER_RABBIT_USER:guest} virtual-host: ${ZIPKIN_RECEIVER_RABBIT_VIRTUAL_HOST:/} useSsl: ${ZIPKIN_RECEIVER_RABBIT_USE_SSL:false} uri: ${ZIPKIN_RECEIVER_RABBIT_URI:} diff --git a/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraConfig.java b/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraConfig.java index 6c1d1a55faf..2839a8af035 100644 --- a/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraConfig.java +++ b/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraConfig.java @@ -28,10 +28,6 @@ public class CassandraConfig extends ModuleConfig { private boolean ensureSchema = true; - /** - * The maximum size of batch size of CQL execution - */ - protected int maxSizeOfBatchCql = 2000; /** * async batch execute pool size */ @@ -93,14 +89,6 @@ public void setPassword(String password) { this.password = password; } - public int getMaxSizeOfBatchCql() { - return maxSizeOfBatchCql; - } - - public void setMaxSizeOfBatchCql(int maxSizeOfBatchCql) { - this.maxSizeOfBatchCql = maxSizeOfBatchCql; - } - public int getAsyncBatchPersistentPoolSize() { return asyncBatchPersistentPoolSize; } diff --git a/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraProvider.java b/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraProvider.java index 9803fb0303b..af87c80fc17 100644 --- a/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraProvider.java +++ b/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/CassandraProvider.java @@ -70,7 +70,7 @@ public class CassandraProvider extends ModuleProvider { @Override public String name() { - return "cassandra"; + return "cassandra3"; } @Override @@ -104,7 +104,7 @@ public void prepare() throws ServiceNotProvidedException, ModuleStartException { new StorageBuilderFactory.Default()); this.registerServiceImplementation( IBatchDAO.class, - new CassandraBatchDAO(client, moduleConfig.getMaxSizeOfBatchCql(), moduleConfig.getAsyncBatchPersistentPoolSize()) + new CassandraBatchDAO(client, moduleConfig.getAsyncBatchPersistentPoolSize()) ); this.registerServiceImplementation( StorageDAO.class, diff --git a/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/dao/CassandraBatchDAO.java b/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/dao/CassandraBatchDAO.java index 45d5d4ab8ba..c14fd9f1650 100644 --- a/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/dao/CassandraBatchDAO.java +++ b/zipkin-server/storage-cassandra/src/main/java/zipkin/server/storage/cassandra/dao/CassandraBatchDAO.java @@ -37,17 +37,15 @@ public class CassandraBatchDAO implements IBatchDAO { private CassandraClient client; private final DataCarrier dataCarrier; - private final int maxBatchSqlSize; private static final ConcurrentHashMap PREPARED_STATEMENT_MAP = new ConcurrentHashMap<>(); - public CassandraBatchDAO(CassandraClient client, int maxBatchCqlSize, int asyncBatchPersistentPoolSize) { + public CassandraBatchDAO(CassandraClient client, int asyncBatchPersistentPoolSize) { this.client = client; String name = "CASSANDRA_ASYNCHRONOUS_BATCH_PERSISTENT"; if (LOG.isDebugEnabled()) { - LOG.debug("CASSANDRA_ASYNCHRONOUS_BATCH_PERSISTENT poolSize: {}, maxBatchCqlSize:{}", asyncBatchPersistentPoolSize, maxBatchCqlSize); + LOG.debug("CASSANDRA_ASYNCHRONOUS_BATCH_PERSISTENT poolSize: {}", asyncBatchPersistentPoolSize); } - this.maxBatchSqlSize = maxBatchCqlSize; this.dataCarrier = new DataCarrier<>(name, asyncBatchPersistentPoolSize, 10000); this.dataCarrier.consume(new CassandraBatchConsumer(this), asyncBatchPersistentPoolSize, 20); } @@ -73,7 +71,7 @@ public CompletableFuture flush(List prepareRequests) { }); if (LOG.isDebugEnabled()) { - LOG.debug("to execute sql statements execute, data size: {}, maxBatchSqlSize: {}", cqls.size(), maxBatchSqlSize); + LOG.debug("to execute sql statements execute, data size: {}", cqls.size()); } if (CollectionUtils.isEmpty(cqls)) { return CompletableFuture.completedFuture(null); @@ -108,7 +106,7 @@ public CompletableFuture flush(List prepareRequests) { if (LOG.isDebugEnabled()) { long end = System.currentTimeMillis(); long cost = end - start; - LOG.debug("execute sync sql statements done, data size: {}, maxBatchSqlSize: {}, cost:{}ms", prepareRequests.size(), maxBatchSqlSize, cost); + LOG.debug("execute sync sql statements done, data size: {}, cost:{}ms", prepareRequests.size(), cost); } return CompletableFuture.completedFuture(null); } diff --git a/zipkin-server/storage-cassandra/src/test/resources/application.yml b/zipkin-server/storage-cassandra/src/test/resources/application.yml index 7bb7cafb15e..e48f82844e7 100644 --- a/zipkin-server/storage-cassandra/src/test/resources/application.yml +++ b/zipkin-server/storage-cassandra/src/test/resources/application.yml @@ -38,7 +38,6 @@ core: serverIdleTimeOut: ${ZIPKIN_SERVER_IDLE_TIMEOUT:30000} serverAcceptQueueSize: ${ZIPKIN_SERVER_QUEUE_SIZE:0} serverMaxRequestHeaderSize: ${ZIPKIN_SERVER_MAX_REQUEST_HEADER_SIZE:8192} - serverThreadPoolQueueSize: ${ZIPKIN_SERVER_POOL_QUEUE_SIZE:-1} serverEnableTLS: ${ZIPKIN_SERVER_SSL_ENABLED:false} serverTLSKeyPath: ${ZIPKIN_SERVER_SSL_KEY_PATH:""} serverTLSCertChainPath: ${ZIPKIN_SERVER_SSL_CERT_CHAIN_PATH:""} @@ -46,8 +45,8 @@ core: searchEnable: ${ZIPKIN_SEARCH_ENABLED:true} storage: - selector: ${ZIPKIN_STORAGE:cassandra} - cassandra: + selector: ${ZIPKIN_STORAGE:cassandra3} + cassandra3: keyspace: ${ZIPKIN_STORAGE_CASSANDRA_KEYSPACE:zipkin} # Comma separated list of host addresses part of Cassandra cluster. Ports default to 9042 but you can also specify a custom port with 'host:port'. contactPoints: ${ZIPKIN_STORAGE_CASSANDRA_CONTACT_POINTS:localhost} @@ -59,8 +58,7 @@ storage: # Max pooled connections per datacenter-local host. maxConnections: ${ZIPKIN_STORAGE_CASSANDRA_MAX_CONNECTIONS:8} # Using ssl for connection, rely on Keystore - use-ssl: ${ZIPKIN_STORAGE_CASSANDRA_USE_SSL:false} - maxSizeOfBatchCql: ${ZIPKIN_STORAGE_CASSANDRA_MAX_SIZE_OF_BATCH_CQL:2000} + useSsl: ${ZIPKIN_STORAGE_CASSANDRA_USE_SSL:false} asyncBatchPersistentPoolSize: ${ZIPKIN_STORAGE_CASSANDRA_ASYNC_BATCH_PERSISTENT_POOL_SIZE:4} receiver-zipkin: diff --git a/zipkin-server/zipkin-storage-ext/zipkin-dependency-storage-cassandra/src/main/java/zipkin/server/dependency/storage/cassandra/ZipkinDependencyCassandraStorageProvider.java b/zipkin-server/zipkin-storage-ext/zipkin-dependency-storage-cassandra/src/main/java/zipkin/server/dependency/storage/cassandra/ZipkinDependencyCassandraStorageProvider.java index f5d3ab56cad..ab203489b02 100644 --- a/zipkin-server/zipkin-storage-ext/zipkin-dependency-storage-cassandra/src/main/java/zipkin/server/dependency/storage/cassandra/ZipkinDependencyCassandraStorageProvider.java +++ b/zipkin-server/zipkin-storage-ext/zipkin-dependency-storage-cassandra/src/main/java/zipkin/server/dependency/storage/cassandra/ZipkinDependencyCassandraStorageProvider.java @@ -35,7 +35,7 @@ public class ZipkinDependencyCassandraStorageProvider extends ModuleProvider { @Override public String name() { - return "cassandra"; + return "cassandra3"; } @Override