Herramienta de anotación Spacy entidades índices

¿Cómo puedo leer mis datos anotados en Spacy?

1) Mi formulario de datos anotados:

"annotation": [ [ 79, 99, "Nom complet" ], 

2) Formulario de datos anotados en el script:

  "annotation": [ { "label": [ "Companies worked at" ], "points": [ { "start": 1749, "end": 1754, "text": "Oracle" } ] }, 

3) ¿Cómo puedo cambiar este código que puede leer mis datos anotados?

 for line in lines: data = json.loads(line) text = data['text'] entities = [] for annotation in data['annotation']: #only a single point in text annotation. point = annotation['points'][0] labels = annotation['label'] # handle both list of labels or a single label. if not isinstance(labels, list): labels = [labels] for label in labels: dataturks indices are both inclusive [start, end] but spacy is not [start, end) entities.append(([0], [1],[2])) training_data.append((text, {"entities" : entities})) 

Training Json: – [{ "text": "This Labor-Contract ('CONTRACT'), effective as of May 12, 2017 (“Effective Date”), is made by and between Client-ABC, Inc. ('Client-ABC'), having its principal place of business at 1030 Client-ABC Street, Atlanta, GA 30318, USA and Supplier-ABC (“Supplier”), having a place of business at 100 Park Avenue, Miami, 10178, USA (hereinafter referred to individually as “Party” and collectively as “Parties”).", "entities": [ [ 50, 62, "EFFECTIVE_DATE" ], [ 106, 116, "VENDOR_NAME" ], [ 181, 203, "VENDOR_ADDRESS" ], [ 205, 212, "VENDOR_CITY" ], [ 214, 216, "VENDOR_STATE" ], [ 217, 222, "VENDOR_POSTAL_CODE" ], [ 224, 227, "VENDOR_COUNTRY" ] ] },{second training data}]

Código de entrenamiento personalizado: –

 training_pickel_file = "training_pickel_file.json" with open(training_pickel_file) as input: TRAIN_DATA = json.load(input) for annotations in TRAIN_DATA: for ent in annotations["entities"]: ner.add_label(ent[2]) other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner'] with nlp.disable_pipes(*other_pipes): # only train NER optimizer = nlp.begin_training() for itn in range(n_iter): random.shuffle(TRAIN_DATA) losses = {} for a in TRAIN_DATA: doc = nlp.make_doc(a["text"]) gold = GoldParse(doc, entities = a["entities"]) nlp.update([doc], [gold], drop =0.5, sgd=optimizer, losses = losses) print('Losses', losses)