...
The normal Avro schema checking will then fail the check as the numerical probability has been replaced by a string.
Extended Schema & Monitoring
To enable monitoring out-of-the-box (OOTB), ModelOp Center introduced in V2.4 the concept of an extended schema. We will go over the details below, but in short, an extended schema is a rich Avro schema, so that more information about the data can be learned. Traditionally, Avro schemas specify field names and types. Extended schemas add more key:value
pairs to each field, so that OOTB monitors can make reasonable assumptions, such as inferring the role of a field ("predictor"
, "identifier"
, etc.)
An Example
Extended schemas are best understood through an example. Let’s consider the same records from the previous example:
...