roboto.experimental.ingestion_rules
Request and response models for an org’s ingestion rules, which decide which uploaded files are ingested and with which action.
A rule has path patterns and the action that ingests matching files. A file whose path a rule matches when it is uploaded is ingestable, and while the rule’s auto_ingest is on, Roboto runs that action on it. There is no client class yet: call the /v1/ingestion-rules endpoints with RobotoClient.
Usage
List the org’s rules:
from roboto.experimental.ingestion_rules import IngestionRuleRecord
from roboto.http import RobotoClient
rules = (
RobotoClient.defaulted()
.get("v1/ingestion-rules")
.to_record_list(IngestionRuleRecord, json_path=["data", "items"])
)Submodules
Package Contents
CreateIngestionRuleRequest
Bases: pydantic.BaseModel
Request payload to create an ingestion rule.
Parameters
data AnyAttributes
CreateIngestionRuleRequest.auto_ingest
CreateIngestionRuleRequest.description
CreateIngestionRuleRequest.exclude_patterns
CreateIngestionRuleRequest.include_patterns
CreateIngestionRuleRequest.invocation
How each matching upload is ingested. Its invocation_input selects only inputs beyond the uploaded file, which the rule’s trigger always passes first. Required when auto_ingest is on.
CreateIngestionRuleRequest.name
DefaultIngestionRuleRecord
Bases: pydantic.BaseModel
Wire format of one of the rules every new org starts with, and whether the caller’s org already has it.
POST /v1/ingestion-rules/defaults creates the ones not already_present.
Parameters
data AnyAttributes
DefaultIngestionRuleRecord.already_present
Whether the org has a rule with this name, or a rule with one of these include patterns.
DefaultIngestionRuleRecord.exclude_patterns
DefaultIngestionRuleRecord.include_patterns
DefaultIngestionRuleRecord.invocation
DefaultIngestionRuleRecord.name
ExistingFilesPreview
Bases: pydantic.BaseModel
Files already uploaded that a rule matches, that are not ingested, and whose current upload no rule’s trigger has run on: ones uploaded, or uploaded again, before the rule existed or while its auto-ingest was off. Editing a file’s metadata or tags does not make it one of these.
Response of GET /v1/ingestion-rules/<rule_id>/existing-files.
Parameters
data AnyAttributes
ExistingFilesPreview.complete_session_count
Complete sessions holding any of these files. After a session’s files are ingested, it is announced as fully ingested again, and the triggers on session.ingested start for it.
ExistingFilesPreview.count
ExistingFilesPreview.count_is_lower_bound
Whether there may be more than count; counting stops after a fixed number of files.
ExistingFilesPreview.sample_paths
Up to 10 of the files’ paths. Empty for a caller who is not an org admin.
IngestFileOutcome
Bases: roboto.compat.StrEnum
What POST /v1/ingestion-rules/ingest did with one file.
Attributes
IngestFileOutcome.AlreadyRunning
An ingestion run on the file is still queued or running, so no new one was started.
IngestFileOutcome.AutoIngestOff
Rules match the file, but none of them auto-ingests.
IngestFileOutcome.NoMatchingRule
No rule in the file’s org matches its path.
IngestFileOutcome.NotFound
No such file in the caller’s org, the caller may not edit it, or it is not in a dataset (rules apply only to dataset files).
IngestFileOutcome.NotStarted
A matching rule’s trigger declined the file (it is disabled, say).
IngestFileResult
Bases: pydantic.BaseModel
!!! abstract “Usage Documentation”
[Models](../concepts/models.md)
A base class for creating Pydantic models.
Attributes
IngestFileResult.__class_vars__
__class_vars__ #The names of the class variables defined on the model.
IngestFileResult.__private_attributes__
__private_attributes__ #Metadata about the private attributes of the model.
IngestFileResult.__signature__
__signature__ #The synthesized __init__ [Signature][inspect.Signature] of the model.
IngestFileResult.__pydantic_complete__
__pydantic_complete__ #Whether model building is completed, or if there are still undefined fields.
IngestFileResult.__pydantic_custom_init__
__pydantic_custom_init__ #Whether the model has a custom __init__ function.
IngestFileResult.__pydantic_decorators__
__pydantic_decorators__ #Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.
IngestFileResult.__pydantic_generic_metadata__
__pydantic_generic_metadata__ #A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.
IngestFileResult.__pydantic_parent_namespace__
__pydantic_parent_namespace__ #Parent namespace of the model, used for automatic rebuilding of models.
IngestFileResult.__pydantic_post_init__
__pydantic_post_init__ #The name of the post-init method for the model, if defined.
IngestFileResult.__pydantic_root_model__
__pydantic_root_model__ #Whether the model is a [RootModel][pydantic.root_model.RootModel].
IngestFileResult.__pydantic_serializer__
__pydantic_serializer__ #The pydantic-core SchemaSerializer used to dump instances of the model.
IngestFileResult.__pydantic_validator__
__pydantic_validator__ #The pydantic-core SchemaValidator used to validate instances of the model.
IngestFileResult.__pydantic_fields__
__pydantic_fields__ #A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.
IngestFileResult.__pydantic_computed_fields__
__pydantic_computed_fields__ #A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.
IngestFileResult.__pydantic_extra__
__pydantic_extra__ #A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.
IngestFileResult.__pydantic_fields_set__
__pydantic_fields_set__ #The names of fields explicitly set during instantiation.
IngestFileResult.__pydantic_private__
__pydantic_private__ #Values of private attributes set on the model instance.
Parameters
data AnyAttributes
IngestFilesRequest
Bases: pydantic.BaseModel
Request payload to ingest existing files now with the org’s rules that match them.
Parameters
data AnyAttributes
IngestFilesRequest.file_ids
IngestFilesResponse
Bases: pydantic.BaseModel
!!! abstract “Usage Documentation”
[Models](../concepts/models.md)
A base class for creating Pydantic models.
Attributes
IngestFilesResponse.__class_vars__
__class_vars__ #The names of the class variables defined on the model.
IngestFilesResponse.__private_attributes__
__private_attributes__ #Metadata about the private attributes of the model.
IngestFilesResponse.__signature__
__signature__ #The synthesized __init__ [Signature][inspect.Signature] of the model.
IngestFilesResponse.__pydantic_complete__
__pydantic_complete__ #Whether model building is completed, or if there are still undefined fields.
IngestFilesResponse.__pydantic_custom_init__
__pydantic_custom_init__ #Whether the model has a custom __init__ function.
IngestFilesResponse.__pydantic_decorators__
__pydantic_decorators__ #Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.
IngestFilesResponse.__pydantic_generic_metadata__
__pydantic_generic_metadata__ #A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.
IngestFilesResponse.__pydantic_parent_namespace__
__pydantic_parent_namespace__ #Parent namespace of the model, used for automatic rebuilding of models.
IngestFilesResponse.__pydantic_post_init__
__pydantic_post_init__ #The name of the post-init method for the model, if defined.
IngestFilesResponse.__pydantic_root_model__
__pydantic_root_model__ #Whether the model is a [RootModel][pydantic.root_model.RootModel].
IngestFilesResponse.__pydantic_serializer__
__pydantic_serializer__ #The pydantic-core SchemaSerializer used to dump instances of the model.
IngestFilesResponse.__pydantic_validator__
__pydantic_validator__ #The pydantic-core SchemaValidator used to validate instances of the model.
IngestFilesResponse.__pydantic_fields__
__pydantic_fields__ #A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.
IngestFilesResponse.__pydantic_computed_fields__
__pydantic_computed_fields__ #A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.
IngestFilesResponse.__pydantic_extra__
__pydantic_extra__ #A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.
IngestFilesResponse.__pydantic_fields_set__
__pydantic_fields_set__ #The names of fields explicitly set during instantiation.
IngestFilesResponse.__pydantic_private__
__pydantic_private__ #Values of private attributes set on the model instance.
Parameters
data AnyAttributes
IngestFilesResponse.results
One result per requested file, in request order.
IngestionPathRule
The path half of an ingestion rule: include and exclude Git wildmatch patterns.
File annotation and the rule’s trigger both decide through matches(), so a file is marked ingestable exactly when the rule’s trigger would accept its upload.
IngestionPathRule.matches()
Whether relative_path matches an include pattern and no exclude pattern, and is not a dotfile.
No rule makes a dotfile ingestable, because triggers never run on dotfiles.
Parameters
relative_path strReturn type
IngestionPathRule.of()
Parameters
include_patterns collections.exclude_patterns collections.Return type
IngestionRuleRecord
Bases: pydantic.BaseModel
Wire format of an ingestion rule: which file paths in an org are ingestable, and how they are ingested.
A file whose path matches any of include_patterns and none of exclude_patterns is marked ingestable when it is created. The rule owns one trigger (trigger_id) that runs invocation on each matching upload while auto_ingest is on.
Parameters
data AnyAttributes
IngestionRuleRecord.auto_ingest
Whether matching uploads are ingested automatically: whether the rule’s trigger is enabled. Turning this off disables the trigger and leaves matching files ingestable.
IngestionRuleRecord.created
IngestionRuleRecord.created_by
IngestionRuleRecord.description
IngestionRuleRecord.exclude_patterns
Git wildmatch patterns. A file matching any of them is not ingestable by this rule.
IngestionRuleRecord.include_patterns
Git wildmatch patterns, e.g. **/*.mcap. A file matching any of them is ingestable.
IngestionRuleRecord.invocation
How each matching upload is ingested. The trigger passes the uploaded file as the first input, followed by whatever invocation.invocation_input selects. None only on a rule that does not auto-ingest.
IngestionRuleRecord.modified
IngestionRuleRecord.modified_by
IngestionRuleRecord.name
IngestionRuleRecord.org_id
IngestionRuleRecord.rule_id
IngestionRuleRecord.trigger_id
The trigger that runs invocation on matching uploads, enabled while auto_ingest is on. Only the rule can enable, disable, edit or delete it. None while the rule has no invocation.
IngestionRun
Bases: pydantic.BaseModel
The latest time an ingestion rule’s trigger ran on a file’s current upload.
Parameters
data AnyAttributes
IngestionRun.detail
Why the trigger could not start the action, when it could not.
IngestionRun.file_id
IngestionRun.invocation_id
The action invocation the run started. None when the trigger could not start one.
IngestionRun.started
IngestionRun.status
IngestionRun.trigger_id
IngestionRunStatus
Bases: roboto.compat.StrEnum
Where one run of a rule’s trigger on a file stands.
Attributes
IngestionRunStatus.Failed
The action failed or was cancelled, or the trigger could not start it.
IngestionRunStatus.Finished
The action finished. The file’s ingestion_status says how much of it was ingested.
LatestIngestionRunsRequest
Bases: pydantic.BaseModel
Request payload to look up the latest ingestion run on each of some files.
Parameters
data AnyAttributes
LatestIngestionRunsRequest.file_ids
LatestIngestionRunsResponse
Bases: pydantic.BaseModel
!!! abstract “Usage Documentation”
[Models](../concepts/models.md)
A base class for creating Pydantic models.
Attributes
LatestIngestionRunsResponse.__class_vars__
__class_vars__ #The names of the class variables defined on the model.
LatestIngestionRunsResponse.__private_attributes__
__private_attributes__ #Metadata about the private attributes of the model.
LatestIngestionRunsResponse.__signature__
__signature__ #The synthesized __init__ [Signature][inspect.Signature] of the model.
LatestIngestionRunsResponse.__pydantic_complete__
__pydantic_complete__ #Whether model building is completed, or if there are still undefined fields.
LatestIngestionRunsResponse.__pydantic_core_schema__
__pydantic_core_schema__ #The core schema of the model.
LatestIngestionRunsResponse.__pydantic_custom_init__
__pydantic_custom_init__ #Whether the model has a custom __init__ function.
LatestIngestionRunsResponse.__pydantic_decorators__
__pydantic_decorators__ #Metadata containing the decorators defined on the model. This replaces Model.__validators__ and Model.__root_validators__ from Pydantic V1.
LatestIngestionRunsResponse.__pydantic_generic_metadata__
__pydantic_generic_metadata__ #A dictionary containing metadata about generic Pydantic models. The origin and args items map to the [__origin__][genericalias.__origin__] and [__args__][genericalias.__args__] attributes of [generic aliases][types-genericalias], and the parameter item maps to the __parameter__ attribute of generic classes.
LatestIngestionRunsResponse.__pydantic_parent_namespace__
__pydantic_parent_namespace__ #Parent namespace of the model, used for automatic rebuilding of models.
LatestIngestionRunsResponse.__pydantic_post_init__
__pydantic_post_init__ #The name of the post-init method for the model, if defined.
LatestIngestionRunsResponse.__pydantic_root_model__
__pydantic_root_model__ #Whether the model is a [RootModel][pydantic.root_model.RootModel].
LatestIngestionRunsResponse.__pydantic_serializer__
__pydantic_serializer__ #The pydantic-core SchemaSerializer used to dump instances of the model.
LatestIngestionRunsResponse.__pydantic_validator__
__pydantic_validator__ #The pydantic-core SchemaValidator used to validate instances of the model.
LatestIngestionRunsResponse.__pydantic_fields__
__pydantic_fields__ #A dictionary of field names and their corresponding [FieldInfo][pydantic.fields.FieldInfo] objects.
LatestIngestionRunsResponse.__pydantic_computed_fields__
__pydantic_computed_fields__ #A dictionary of computed field names and their corresponding [ComputedFieldInfo][pydantic.fields.ComputedFieldInfo] objects.
LatestIngestionRunsResponse.__pydantic_extra__
__pydantic_extra__ #A dictionary containing extra values, if [extra][pydantic.config.ConfigDict.extra] is set to ‘allow’.
LatestIngestionRunsResponse.__pydantic_fields_set__
__pydantic_fields_set__ #The names of fields explicitly set during instantiation.
LatestIngestionRunsResponse.__pydantic_private__
__pydantic_private__ #Values of private attributes set on the model instance.
Parameters
data AnyAttributes
LatestIngestionRunsResponse.runs
The latest run per file id. A file whose current upload no rule’s trigger has run on, or that the caller may not view, is absent.
MAX_INGEST_FILES
Most files one POST /v1/ingestion-rules/ingest takes.
MAX_MATCH_PATHS
The most paths one MatchIngestionRulesRequest may ask about.
MatchIngestionRulesRequest
Bases: pydantic.BaseModel
Request payload to ask which of an org’s ingestion rules match each of some file paths.
Parameters
data AnyAttributes
MatchIngestionRulesRequest.paths
File paths relative to their dataset, e.g. logs/run_1.mcap.
MatchIngestionRulesResponse
Bases: pydantic.BaseModel
The rules matching each requested path, oldest rule first. A path no rule matches maps to an empty list.
Parameters
data AnyAttributes
MatchIngestionRulesResponse.matches
UpdateIngestionRuleRequest
Bases: pydantic.BaseModel
Request payload to change an ingestion rule. Only the fields present in the payload change.
Parameters
data AnyAttributes
UpdateIngestionRuleRequest.auto_ingest
Turning this on enables the rule’s trigger, creating it if the rule had none; turning it off disables it.
UpdateIngestionRuleRequest.description
UpdateIngestionRuleRequest.exclude_patterns
UpdateIngestionRuleRequest.include_patterns
UpdateIngestionRuleRequest.invocation
Replaces the rule’s whole invocation when present.