Apache SkyWalking – RESTFul

Blog: Introduce R3 to recognition RESTFul URI

Thu, 17 Oct 2024 00:00:00 +0000

Background

In modern applications, services are typically provided through RESTFul HTTP URIs. Using RESTFul HTTP URIs (as unique resource identifiers) offers high readability, making it easier for both clients and servers to understand. However, in the observability field, this approach poses several challenges:

A large number of endpoints (HTTP URI): Browsing through all externally provided endpoints becomes more difficult, making it hard to identify problematic endpoints.
Metrics are difficult to collect: It becomes particularly challenging to categorize similar endpoints and generate observability metrics.

In existing solutions, this issue can be resolved following these application-level resolutions:

Agent Detection: In certain frameworks, rules are often declared to handle RESTFul requests. For example, in Java’s Spring Web, annotations like @GET can be used, which can then be linked to current requests using a Java Agent.
OpenAPI: Predefined files can be associated with the application, allowing the observability system to be aware of the URIs that may be used.

Both resolutions are tightly coupled with application settings, which can be limiting for unknown applications or applications where the agent cannot be monitored. Therefore, we need to consider whether there is a more general solution to identify URIs and merge metrics generated from similar URIs for better representation.

R3

R3(RESTFul Pattern Recognition) is a high-performance RESTFul URI recognition tool inspired by Drain3. It can be deployed as a standalone application on the observability server and communicate with the SkyWalking OAP.

R3 can accept a URI list via the gRPC protocol and aggregate similar URIs into a specific format. The aggregated (formatted) URI list can also be queried using the gRPC protocol.

Data Interaction Flow

OAP receives and caches unformatted URI list: OAP receives observability data through different protocols and identifies all unformatted URIs. These URIs are stored in a temporary list categorized by the service they belong to.
OAP sends URIs to be formatted to R3: OAP periodically batches the URIs that need formatting and sends them to the R3 service.
R3 receives and parses the URI list: R3 asynchronously analyzes the similarity of the received URIs and stores (persists) the results on the local disk to allow features like recovery after a restart.
OAP queries formatted URI list from R3: OAP periodically queries R3 for the detected formatted URIs and saves the results in memory.
OAP formats URIs: When OAP receives new observability data, it matches the URIs against the formatted URIs retrieved from R3. If a match is found, the formatted URI is used for subsequent metric calculations.

Scenarios

In R3, the following scenarios are primarily addressed. For URIs identified as duplicates, R3 would replace the variable parts with {var} to standardize them.

ID Matching

A common practice in RESTFul APIs is to include various IDs in the URI paths, which leads to a large number of unique URI endpoints. For example, paths like the following will be aggregated by R3 into a standardized format: /api/users/{var}.

/api/users/cbf11b02ea464447b507e8852c32190a
/api/users/5e363a4a18b7464b8cbff1a7ee4c91ca
/api/users/44cf77fc351f4c6c9c4f1448f2f12800
/api/users/38d3be5f9bd44f7f98906ea049694511
/api/users/5ad14302e7924f4aa1d60e58d65b3dd2

Word Detection

In RESTFul URIs, operations on an entity are usually specified using HTTP methods, but often additional types are needed. This is addressed by including specific nouns in the path. To handle this, R3 implements word parsing: when R3 detects specific words in the path, it will not format that part. For example, URIs like the following would not be considered similar and therefore will not be merged:

/api/sale
/api/product_sale
/api/ProductSale

Low Sample

To prevent incorrect judgments due to insufficient sample sizes, R3 allows the configuration of a combine min URI count parameter in the configuration file. This parameter sets the minimum number of similar paths required before proceeding with the analysis.

Such as the threshold is 3, the following URI would keep the original URI, not parameterized.

/api/fetch1
/api/fetch2

But the following URI would be parametrized to /api/{var}, since the sample count is bigger than the threshold.

/api/fetch1
/api/fetch2
/api/fetch3

Version API

In real-world scenarios, we often encounter URIs with multiple versions. R3 addresses this by ensuring that if a specified path contains a v\\d+ parameter (indicating version information), that part would not be parameterized. For example, the following URIs will be separately parsed into /test/v1/{var} and /test/v999/{var}.

/test/v1/cbf11b02ea464447b507e8852c32190a
/test/v1/5e363a4a18b7464b8cbff1a7ee4c91ca
/test/v1/38d3be5f9bd44f7f98906ea049694511
/test/v999/1
/test/v999/2
/test/v999/3

Demo

Next, let’s quickly demonstrate how to use R3 to format observed endpoints, so you can understand more specifically what it accomplishes.

Deploy SkyWalking Showcase

SkyWalking Showcase contains a complete set of example services and can be monitored using SkyWalking. For more information, please check the official documentation.

In this demo, we only deploy service, the latest released SkyWalking OAP, R3 service and UI.

export FEATURE_FLAGS=java-agent-injector,single-node,elasticsearch,r3
make deploy.kubernetes

After deployment is complete, please run the following script to open SkyWalking UI: http://localhost:8080/.

kubectl port-forward svc/ui 8080:8080 --namespace default

Trigger RESTFul Requests

In R3, a scheduled task is started by default to generate RESTFul traffic at regular intervals. However, you can also manually trigger this process using the following command:

kubectl exec -n sample-services $(kubectl get pod -n sample-services --selector=app=gateway -o jsonpath='{.items[0].metadata.name}') -- /bin/bash -c 'for i in $(seq 1 200); do curl http://rating/songs/$i/reviews/$((i+1)); sleep 1; done'

In the above command, R3 would automatically locate the gateway node and send requests in RESTFul format to the rating service within that node. This allows R3 to generate and test traffic patterns that simulate real RESTFul requests to the target service.

Check Formatted URIs

Once the RESTFul requests are triggered, you can view the aggregated endpoints in the UI.

Note: Since the formatted endpoints are generated asynchronously, some of the earlier requests may not yet be formatted. You may need to wait for some time before the UI shows only the formatted addresses.

Conclusion

In this article, we discussed in detail how SkyWalking utilizes the R3 service to format RESTFul URIs and aggregate related metrics upon receiving them. Currently, it applies to most RESTFul scenarios, and if more cases need to be supported, we can extend it further as needed.

Zh: 引入 R3 识别 RESTFul URI

Thu, 17 Oct 2024 00:00:00 +0000

背景

在现代应用中，服务通常通过 RESTFul HTTP URI 提供。使用 RESTFul HTTP URI 作为唯一的资源标识符，不仅具备良好的可读性，还能让客户端和服务器更容易理解请求。然而，在可观测性领域，这种方式也带来了一些挑战：

大量的端点（HTTP URI）：浏览所有对外提供的端点变得更加困难，因此很难识别出存在问题的端点。
指标收集困难：尤其难以对类似的端点进行归类并生成可观测性指标。

现有解决方案通常采用以下应用级别的方式来解决此问题：

代理检测：在某些框架中，通常会声明规则来处理 RESTFul 请求。例如，在 Java 的 Spring Web 中可以使用 @GET 等注解，然后可以通过 Java Agent 将其与当前请求关联起来。
OpenAPI：应用可以关联预定义文件，使可观测性系统知晓可能使用的 URI。

这两种解决方案都与应用设置紧密耦合，这对于未知应用或者无法监控代理的应用来说是一个局限。因此，我们需要考虑是否有一种更通用的解决方案来识别 URI，并合并来自类似 URI 生成的指标，以便更好地展示数据。

R3

R3（RESTFul 模式识别）是一个高性能的 RESTFul URI 识别工具，其灵感来自 Drain3。它可以作为独立应用部署在可观测性服务器上，并与 SkyWalking OAP 进行通信。

R3 可以通过 gRPC 协议接收 URI 列表，并将类似的 URI 聚合为特定格式。聚合后的（格式化）URI 列表也可以通过 gRPC 协议进行查询。

数据交互流程

OAP 接收并缓存未格式化的 URI 列表：OAP 通过不同协议接收可观测性数据，并识别出所有未格式化的 URI。然后将这些 URI 按照所属服务暂时存储在列表中。
OAP 将待格式化的 URI 发送给 R3：OAP 定期将需要格式化的 URI 批量发送到 R3 服务。
R3 接收并解析 URI 列表：R3 异步分析接收到的 URI 相似性，并将结果存储（持久化）在本地磁盘中，以便在重启后进行恢复。
OAP 查询 R3 中的格式化 URI 列表：OAP 定期查询 R3 中检测到的格式化 URI，并将结果保存在内存中。
OAP 格式化 URI：当 OAP 接收到新的可观测性数据时，会将 URI 与从 R3 获取的格式化 URI 进行匹配。如果匹配成功，后续的指标计算将使用格式化后的 URI。

场景

在 R3 中，主要解决以下场景。对于识别为重复的 URI，R3 将用 {var} 替换变量部分以标准化 URI。

ID 匹配

在 RESTFul API 中，常见做法是将各种 ID 包含在 URI 路径中，这导致了大量唯一的 URI 端点。例如，以下路径将被 R3 聚合为标准化格式 /api/users/{var}。

/api/users/cbf11b02ea464447b507e8852c32190a
/api/users/5e363a4a18b7464b8cbff1a7ee4c91ca
/api/users/44cf77fc351f4c6c9c4f1448f2f12800
/api/users/38d3be5f9bd44f7f98906ea049694511
/api/users/5ad14302e7924f4aa1d60e58d65b3dd2

词语检测

在 RESTFul URI 中，实体的操作通常通过 HTTP 方法指定，但有时还需要在路径中添加特定名词。为此，R3 实现了词语解析：当 R3 检测到路径中的特定词语时，将不会格式化该部分。例如，以下 URI 将不会被视为相似，因此不会被合并：

/api/sale
/api/product_sale
/api/ProductSale

样本不足

为防止由于样本不足而导致错误判断，R3 允许在配置文件中配置最小 URI 合并计数参数。

例如，当阈值为 3 时，以下 URI 将保持原样，不会被参数化。

/api/fetch1
/api/fetch2

但是以下 URI 将被参数化为 /api/{var}，因为样本数大于阈值。

/api/fetch1
/api/fetch2
/api/fetch3

版本 API

在实际场景中，常会遇到包含多个版本的 URI。R3 通过确保指定路径中包含 v\\d+ 参数（表示版本信息）来解决这个问题，该部分将不会被参数化。例如，以下 URI 将分别解析为 /test/v1/{var} 和 /test/v999/{var}。

/test/v1/cbf11b02ea464447b507e8852c32190a
/test/v1/5e363a4a18b7464b8cbff1a7ee4c91ca
/test/v1/38d3be5f9bd44f7f98906ea049694511
/test/v999/1
/test/v999/2
/test/v999/3

演示

接下来我们快速演示如何使用 R3 格式化观察到的端点，帮助你更具体地理解它的功能。

部署 SkyWalking Showcase

SkyWalking Showcase 包含一整套示例服务，可以通过 SkyWalking 进行监控。更多信息请查看官方文档。

在此演示中，我们仅部署服务、最新发布的 SkyWalking OAP、R3 服务和 UI。

export FEATURE_FLAGS=java-agent-injector,single-node,elasticsearch,r3
make deploy.kubernetes

部署完成后，请运行以下脚本打开 SkyWalking UI：http://localhost:8080/。

kubectl port-forward svc/ui 8080:8080 --namespace default

触发 RESTFul 请求

在 R3 中，默认启动了定时任务以定期生成 RESTFul 流量。不过，你也可以使用以下命令手动触发此过程：

kubectl exec -n sample-services $(kubectl get pod -n sample-services --selector=app=gateway -o jsonpath='{.items[0].metadata.name}') -- /bin/bash -c 'for i in $(seq 1 200); do curl http://rating/songs/$i/reviews/$((i+1)); sleep 1; done'

在上述命令中，R3 将自动定位网关节点，并以 RESTFul 格式向该节点内的 rating 服务发送请求。此操作允许 R3 生成并测试模拟实际 RESTFul 请求的流量模式。

查看格式化 URI

一旦触发 RESTFul 请求，你可以在 UI 中查看聚合后的端点。

注意：由于格式化端点是异步生成的，因此一些较早的请求可能尚未被格式化。你可能需要等待一段时间，UI 才会仅显示格式化后的地址。

结论

在本文中，我们详细讨论了 SkyWalking 如何利用 R3 服务来格式化 RESTFul URI，并在接收后聚合相关的指标。目前，它适用于大多数 RESTFul 场景，如果需要支持更多情况，我们可以根据需求进一步扩展。