field.exclude
Remove a subset of fields from the record.
Description
Remove a subset of fields from the record, all the other fields are left untouched.
If a field is excluded that contains nested data, the whole tree will be removed.
It is not allowed to exclude .Position or .Operation fields.
Note that this processor only runs on structured data, if the record contains
raw JSON data, then use the processor json.decode
to parse it into structured data first.
Configuration parameters
| Name | Type | Default | Description |
|---|---|---|---|
fields | string | null | Fields is a comma separated list of target fields which should be excluded. For more information about the format, see Referencing fields. |
Examples
Exclude all fields in payload
Excluding all fields in .Payload results in an empty payload.
Configuration parameters
| Name | Value |
|---|---|
fields | .Payload |
Record difference
Before | After | ||||
1 | { | 1 | { | ||
2 | "position": null, | 2 | "position": null, | ||
3 | "operation": "create", | 3 | "operation": "create", | ||
4 | "metadata": { | 4 | "metadata": { | ||
5 | "key1": "val1" | 5 | "key1": "val1" | ||
6 | }, | 6 | }, | ||
7 | "key": null, | 7 | "key": null, | ||
8 | "payload": { | 8 | "payload": { | ||
9 | - | "before": { | 9 | + | "before": null, |
10 | - | "bar": "baz" | 10 | + | "after": null |
11 | - | }, | |||
12 | - | "after": { | |||
13 | - | "foo": "bar" | |||
14 | - | } | |||
15 | } | 11 | } | ||
16 | } | 12 | } | ||
Exclude multiple fields
It's possible to exclude multiple fields by providing a
comma-separated list of fields. In this example, we exclude .Metadata,
.Payload.After.foo and .Key.key1.
Configuration parameters
| Name | Value |
|---|---|
fields | .Metadata,.Payload.After.foo,.Key.key1 |
Record difference
Before | After | ||||
1 | { | 1 | { | ||
2 | "position": null, | 2 | "position": null, | ||
3 | "operation": "create", | 3 | "operation": "create", | ||
4 | - | "metadata": { | 4 | + | "metadata": {}, |
5 | - | "source": "s3" | |||
6 | - | }, | |||
7 | "key": { | 5 | "key": { | ||
8 | - | "key1": "val1", | |||
9 | "key2": "val2" | 6 | "key2": "val2" | ||
10 | }, | 7 | }, | ||
11 | "payload": { | 8 | "payload": { | ||
12 | "before": { | 9 | "before": { | ||
13 | "bar": "baz" | 10 | "bar": "baz" | ||
14 | }, | 11 | }, | ||
15 | "after": { | 12 | "after": { | ||
16 | - | "foo": "bar", | |||
17 | "foobar": "baz" | 13 | "foobar": "baz" | ||
18 | } | 14 | } | ||
19 | } | 15 | } | ||
20 | } | 16 | } | ||