avro.decode

Decodes a field's raw data in the Avro format.

Description

The processor takes raw data (bytes or a string) in the specified field and decodes it from the Avro format into structured data. It extracts the schema ID from the data, downloads the associated schema from the schema registry and decodes the payload. The schema is cached locally after it's first downloaded.

If the processor encounters structured data or the data can't be decoded it returns an error.

This processor is the counterpart to avro.encode.

Configuration parameters

Name	Type	Default	Description
auth.basic.password	string	null	The password to use with basic authentication. This option is required if auth.basic.username contains a value. If both auth.basic.username and auth.basic.password are empty basic authentication is disabled.
auth.basic.username	string	null	The username to use with basic authentication. This option is required if auth.basic.password contains a value. If both auth.basic.username and auth.basic.password are empty basic authentication is disabled.
field	string	.Payload.After	The field that will be decoded. For more information about the format, see Referencing fields.
tls.ca.cert	string	null	The path to a file containing PEM encoded CA certificates. If this option is empty, Conduit falls back to using the host's root CA set.
tls.client.cert	string	null	The path to a file containing a PEM encoded certificate. This option is required if tls.client.key contains a value. If both tls.client.cert and tls.client.key are empty TLS is disabled.
tls.client.key	string	null	The path to a file containing a PEM encoded private key. This option is required if tls.client.cert contains a value. If both tls.client.cert and tls.client.key are empty TLS is disabled.
url	string	null	URL of the schema registry (e.g. http://localhost:8085)

Examples

Decode a record field in Avro format

This example shows the usage of the avro.decode processor. The processor decodes the record's.Key field using the schema that is downloaded from the schema registry and needs to exist under the subjectexample-decode. In this example we use the following schema:

{
  "type":"record",
  "name":"record",
  "fields":[
    {"name":"myString","type":"string"},
    {"name":"myInt","type":"int"}
  ]
}

Configuration parameters

Name	Value
auth.basic.password	null
auth.basic.username	null
field	.Key
tls.ca.cert	null
tls.client.cert	null
tls.client.key	null
url	http://127.0.0.1:54322

Record difference

Before			After
1		{	1		{
2		"position": "dGVzdC1wb3NpdGlvbg==",	2		"position": "dGVzdC1wb3NpdGlvbg==",
3		"operation": "create",	3		"operation": "create",
4		"metadata": {	4		"metadata": {
5		"key1": "val1"	5		"key1": "val1"
6		},	6		},
7	-	"key": "\u0000\u0000\u0000\u0000\u0001\u0006bar\u0002",	7	+	"key": {
			8	+	"myInt": 1,
			9	+	"myString": "bar"
			10	+	},
8		"payload": {	11		"payload": {
9		"before": null,	12		"before": null,
10		"after": null	13		"after": null
11		}	14		}
12		}	15		}