feat: multi modal document by alaeddine-13 · Pull Request #188 · docarray/docarray

alaeddine-13 · 2022-03-10T08:18:21Z

feat: multi modal document

add type annotations
add from_dataclass method
store schema in root document meta tags (possible also move other attributes to meta tags)
build a sub attribute accessor using schema stored in root document
extend traversal path syntax
protobuf serialization/deserialization
modifications to traversal path grammar:

[] for attributes WARNING: this entails that the grammer becomes ambiguous
[] optional for slice
tolerate white space
accept offset index not just slice

test protobuf serialization/deserialization
extend dataclass decorator + add method from_document to decorated classes [optional]
test coverage

Usage:
create schema:

from docarray.types import dataclass, Text, Image, Audio
@dataclass
class MMDocument:
    title: Text
    image: Image
    version: int

create mm_doc:

obj = MMDocument(title='hello world', image='img.png'), version=20)

translate to Document:

doc = Document.from_dataclass(obj)

traverse a DocumentArray of mm_docs:

da = DocumentArray([obj, ...])
title_docs = da['@r.[title]']
image_docs = da['@r.[image]']
image_title_docs = da['@r.[image,title]']

traverse with slice:

da = DocumentArray([obj, ...])
first_3_title_docs = da['@r[:3].[title]']
first_image_docs = da['@r[1].[image]']
last_image_title_docs = da['@r[-1].[image,title]']

Note: the syntax of traversal paths is extended, there should be no breaking changes (e.g, old r1:5c:3 syntax is still accepted)

codecov · 2022-03-10T08:22:59Z

Codecov Report

Merging #188 (70e565d) into main (60d1665) will increase coverage by 0.39%.
The diff coverage is 90.84%.

@@            Coverage Diff             @@
##             main     #188      +/-   ##
==========================================
+ Coverage   84.91%   85.30%   +0.39%     
==========================================
  Files         119      123       +4     
  Lines        5383     5677     +294     
==========================================
+ Hits         4571     4843     +272     
- Misses        812      834      +22

Flag	Coverage Δ
docarray	`85.30% <90.84%> (+0.39%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
docarray/types/__init__.py	`10.52% <0.00%> (ø)`
docarray/types/deserializers.py	`73.91% <73.91%> (ø)`
docarray/document/mixins/_property.py	`85.80% <85.71%> (+2.02%)`	⬆️
docarray/types/multimodal.py	`88.54% <88.54%> (ø)`
docarray/document/mixins/multimodal.py	`91.13% <91.13%> (ø)`
docarray/types/serializers.py	`97.05% <97.05%> (ø)`
docarray/array/mixins/traverse.py	`95.86% <98.30%> (+1.42%)`	⬆️
docarray/document/data.py	`91.48% <100.00%> (+0.09%)`	⬆️
docarray/document/mixins/__init__.py	`100.00% <100.00%> (ø)`
docarray/document/pydantic_model.py	`95.83% <100.00%> (+0.08%)`	⬆️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 60d1665...70e565d. Read the comment docs.

winstonww

I also wonder what's the best way to handle optional parameters? For instance, for img: Optional[ImageDocument], should we append an empty chunk document or not in this case?

alaeddine-13

also add test about support of default values in dataclass and see if it works

alaeddine-13 · 2022-03-22T16:58:23Z

+                }
+
+            elif isinstance(field.type, typing._GenericAlias):
+                if field.type._name in ['List', 'Iterable']:


I find it better to stick to the type that the user explicitly provided.
If we rely on the dynamic type, we will introduce an implicit behaviour to the user

github-actions · 2022-03-22T19:33:40Z