Release Notes

  • NEW Release - 1.10.0

    April 23, 2018

    Entity Extraction and Linking

    • New deep neural network processor (in BETA): We've added an alternative entity extraction processor, which can be used in place of the standard statistical extractor. The new processor employs a deep neural network that improves accuracy up to 7% and error rate up to 32%. It is available for English, Arabic, and Korean. To enable this processor, provide DNN for the modelType. Example: {"content": "your_text_here", "options": {"modelType": "DNN"}}

    Morphological Analysis

    • New language support: We've added support for lemmatizing Catalan, Estonian, Serbian, and Slovak text.

    • Bug fix: Previously, tokens could be empty or contain only invisible characters. Such tokens will no longer be returned.

    Language Identification

    • New language support: Short string language identification now also supports Malay and Indonesian. Both languages were already supported for longer texts.

    Sentiment Analysis

    • New language support: We've added support for sentiment analysis in Persian (Farsi and Dari) at both the document and the entity level.

    Name Translation

    • New language support: Rosette now supports transliteration of person, organization, and location names from Greek to Latin script.

    Name Similarity

    • New language support: Rosette now supports matching of Greek names written in Greek script to English names written in Latin script and other Greek names written in Greek script.

    • Accuracy improvements: We have improved match scores and segmentation rules for Arabic, Western Farsi, and Japanese names.

  • Point Release - 1.9.4

    March 27, 2018

    Morphological Analysis

    • Bug fix: We’ve improved our handling of tokens consisting of numbers and Latin characters, such as serial numbers, in Korean. Previously these tokens were decompounded into multiple morphemes.

    • Bug fix: We’ve added the lower case Russian word “интернет” (“internet”) to the dictionary, which was previously only present in title case.

    • Bug fix: We’ve improved our handling of tokens containing an apostrophe immediately followed by a digit in languages like French and Italian, like “all'M5S”. Previously, the apostrophe would be parsed as its own token.

    • Bug fix: We’ve improved our German analysis by taking better advantage of context clues, and now return more accurate results, especially for uncommon words.

    • Bug fix: We’ve improved our handling of English and German words in all-caps. Previously, these words were assumed to be proper nouns, even though all-caps may simply denote emphasis.

    Transliteration

    • Performance improvement: We’ve added caching of model objects to prevent OutOfMemoryErrors.

    Entity Extraction and Linking

    • Hebrew entity extraction: We’ve added support for entity type "Title" in Hebrew.

  • Point Release - 1.9.3

    February 27, 2018

    Morphological Analysis

    • German disambiguation: We’ve improved our German disambiguator for lemmas and part-of-speech tags to be more sensitive to capitalization, particularly for single word inputs.

    • Bug fix: Previously, German definite articles (der, die, das, den, dem, and des) meaning the were lemmatized inconsistently. They are now all lemmatized to the masculine singular nominative form, der.

    • Bug fix: In some languages, an apostrophe may mark a token boundary, like in the Italian phrase all'M5S. Previously the token boundary was incorrectly omitted when the following token contained a digit. This issue has been rectified and M5S will be properly tokenized.

    Name Similarity

    • Farsi Name Matching: We’ve improved the behavior of Western Farsi-English matching by tokenizing input names earlier in the analysis process.

  • Point Release - 1.9.2

    February 6, 2018

    Entity Extraction and Linking

    • Currency support: We’ve added several additional currency symbols to the regex, including the Turkish Lira (₺), the Pound Sterling (₤), and the Euro (€).

    • Bug fix: We’ve fixed a bug that caused hexadecimal number strings to be incorrectly extracted as products.

    Name Translation

    • Thai improvement: We’ve modified the gemination rules (consonant elongation) of the ISO11940-2 Thai transliteration standard to improve Thai translation accuracy.

    Name Similarity

    • Bug fix: We’ve fixed a bug around matching names of organizations and locations that contain numbers, such as "Century 21 Real Estate LLC." These non-person names containing digits will now match more accurately.

  • NEW Release - 1.9.0

    January 16, 2018

    Topics

    • Salience scores: We've added salience scores for keyphrases and concepts to indicate how relevant an extracted concept or keyphrase is to the overall content of a text. You now have the option to filter out results below a desired threshold value: {"content": "your_text_here", "options": {"keyphraseSalienceThreshold": value, "conceptSalienceThreshold": other_value}}. Read more and try it out!

    • Short string support: We've improved our concept extraction logic, and now support concept extraction for short input strings, i.e. texts less than 280 characters long.

    Name Deduplication

    • Thai support: Rosette now supports deduplication of Thai names.

    Sentiment Analysis

    • New feature: We've added the option to use an experimental alternative deep neural network (DNN) sentiment model for English: {"content": "your_text_here", "options": {"modelType": "DNN"}}. The new model will produce different results, which may be more accurate than the current support vector machine (SVM) model, depending on your data. As it is experimental, we are particularly interested in getting user feedback. Let us know what you think! On-premise users of Rosette API should review the new system requirements in install-guide.txt before using this option. Read more and try it out!

    Entity Extraction and Linking

    • Entity offsets returned: Entity mention offsets are now returned by default. Offsets can be used to locate the exact surface forms of an extracted entity in the document text. See the output sample and try it out!

    • Korean improvements: We’ve significantly improved the accuracy of entity extraction results across all entity types in Korean.

    • Confidence scores: Confidence scores for entities extracted using Rosette’s statistical processor, as well as all linked entities, will now be returned by default. Confidence scores allow Rosette to return the most accurate results, particularly for entity linking. To change this behavior, set {"content": "your_text_here", "options": {"calculateConfidence": false}}

    Language Identification

    • Detect language regions in multilingual documents: The language identification endpoint can now detect different language regions in a multilingual document. Read more and try it out!

    • Score changes: We’ve rescaled the confidence scores returned by the language identification endpoint based on customer feedback. The ranking of language candidates will not change, but the scores themselves will be higher. If you currently filter language identification results based on a confidence threshold, you will need to reset that threshold to maintain parity with previous versions.

    Name Translation

    • Thai support: Rosette now supports transliteration of names from Thai to Latin script.

    Name Similarity

    • Thai support: Rosette now supports matching of Thai names to English names and other Thai names.

    • Accuracy improvements: We have improved match scores for Arabic names (persons, locations and organizations) as well as for Chinese and Japanese organizations.

  • Point Release - 1.8.1

    November 13, 2017

    Entity Extraction and Linking

    • Bug Fix: This release addresses a bug whereby entity linking confidence scores were not being returned when requested. Confidence scores for entities resolved to Wikipedia entries will now be returned when using the following option: {"content": "your_text_here", "options": {"calculateConfidence": true}}

  • NEW Release - 1.8.0

    October 23, 2017

    Topic Extraction

    • New Endpoint: Topic extraction We've added a topic extraction endpoint that identifies the key ideas of an input text. For a given input, the endpoint will return two lists: Keyphrases, a list of phrases extracted directly from the text, and Concepts, a list of phrases which do not have to be explicitly mentioned in the input. Read more and try it out!

    LABS

    • LABS graduates: The /transliteration, /relationships, and /syntax/dependencies endpoints have graduated from “Labs” status and are now fully supported.

    Sentiment Analysis

    • New language: Rosette now supports document and entity-level sentiment analysis in French.

    Entities

    • Salience Scoring: Rosette can now return salience scores, which indicate whether an entity is important to the overall scope of the document. Turn on the scores by adding an option to the request: {"content": "your_text_here", "options": {"calculateSalience": true}}

    • Linking Confidence Scoring: Rosette can also now return Linking Confidence scores, which represent the degree of certainty of the link between an in-document entity mention and its linked QID. It may be used for thresholding and removal of false positives. Linking Confidence scores for entities identified by our linker and assigned with a QID are now available by adding an option to the request: {"content": "your_text_here", "options": {"calculateConfidence": true}}

  • Point Release - 1.7.3

    July 26, 2017

    Name Deduplication

    • New Endpoint: Name Deduplication We've added a name deduplication endpoint that identifies similar names within a list. The endpoint accepts a list of names, organizes the list into clusters of unique names, and assigns each cluster with an id number. It then returns those ids to the user. Read more and try it out!

  • Point Release - 1.7.2

    June 22, 2017

    Entity Extraction

    • Bug Fix: This release addresses a backward compatibility issue between the latest Rosette API and older versions of our Java binding that affected Rosette's ability to return entity confidence scores. Confidence scores for entities identified by our statistical extractor are now available by adding an option to the request: {"content": "your_text_here", "options": {"calculateConfidence": true}}

  • NEW Release - 1.7.1

    June 14, 2017

    Transliteration for Arabizi

    • New Endpoint: Transliteration We've added a transliteration endpoint that converts between Arabic written in ASCII, also called Romanized Arabic chat or Arabizi, and native Arabic script.

    Arabic Sentiment Analysis

    • Beta Arabic Support for /sentiment We now return document-level and entity-level sentiment analysis results for Arabic language input.

    Relationship Extraction

    • Personal pronoun resolution for /relationships Building on the pronoun resolution capabilities of our /entities endpoint, pronouns which are resolved to named entities can now be arguments in relationships.

    Entities

    • Improved Confidence Scoring: Confidence score calculation is improved to correlate well with precision and may be used for thresholding and removal of false positives.

    Tokenization

    • New support for emoticons, emoji, @mentions, hashtags, URLs, and email addresses These special characters and character combinations are now kept together as a single token in all languages, greatly improving the accuracy of analysis further downstream.

    Morphological Analysis

    • Improved accuracy for English and Spanish For this release, we updated our English and Spanish dictionaries. We also introduced new, more advanced disambiguation models for these languages, which help Rosette to correctly determine a given word’s part of speech. For example, words like “object” can be either a noun (“this is an object”) or a verb (“I object!”).

    • Lemmatization and normalization of emoticons, emoji, @mentions, hashtags, URLs, and email addresses Rosette now normalizes and lemmatizes these special characters and character combinations to streamline analysis.

    • Improved decompounding for Dutch Dutch language text is now decompounded more accurately, Dutch text is now decompounded more accurately, producing better tokens for search enhancement and other applications.

  • NEW Release - 1.6.0

    March 23, 2017

    Relationship Extraction

    • Improved Accuracy of Corporate Relationships: Improvements made to the identification of relationships between corporations. The relationships involved are: ORG-SUBSIDIARY-OF, ORG-COLLABORATORS, ORG-ACQUIRED-BY and ORG-PROVIDER-TO.

    • Removed the ORG-PARTNERSHIPS Relationship: The ORG-PARTNERSHIPS relationship is now subsumed under ORG-COLLABORATORS and is no longer extracted as an independent relationship.

    Entity Extraction and Linking

    • Improved Linking Accuracy via Inclusion of New Context Features: The statistical model for entity linking includes features that measure the vector space similarity between an entity context and the Wikipedia contexts of its potential linking targets. The new features result in higher F-Scores across all supported languages.

    • Entity Linking in Japanese, Chinese and Spanish: Entity linking to Wikidata with QIDs for Japanese, Chinese and Spanish text is supported.

    • Removed Long Text Linking: Entity linking to Wikidata (with QIDs) for long texts is removed, which, as a result, removed entity linking capabilities in Arabic.

    Text Embedding

    • Vector dimension reduced from 512 to 300: We are able to produce smaller vectors that are more efficient and memory friendly without sacrificing overall speed or accuracy.

    • Improved Speed and Accuracy: A number of speed enhancements have been made along with much larger vocabularies to increase accuracy.

    Language Identification

    • Improved Accuracy on Texts with Mixed Scripts: A script specific model is now selected based on the weighted frequency of the different scripts in the input.

    Name Matching

    • Japanese Improvements: Rosette API now has better support for Japanese name matching. This includes the new use of word embeddings, which are used to match words with similar semantic meaning, as well as improved Japanese name segmentation.
  • NEW Release - 1.5.1

    January 10, 2017

    Targeted Relationship Extraction

    • New Endpoint Functionality: The /relationships endpoint now returns targeted relationships, as opposed to the former open relationships, as its default extracted relationships. Targeted relationships are specifically between two entities, and are labeled by a certain relationship type. You can see the former open relationships by setting the option of "discoveryMode" to "true". Read more and try it out!

    /entities/linked REMOVED

    • Removed Deprecated Endpoint: The /entities/linked endpoint, previously deprecated, is now completely removed. All functionality is available through the /entities endpoint. You will receive a 404 when calling /entities/linked.

    Entity Extraction

    • Social Media Linking in Japanese and Chinese: Our fast short text entity linker to Wikidata is now available for Japanese and Chinese.
    • Removal of long text entity linking: Our long text entity linker has been replaced by our fast short string entity linker. You will now see entity linking results from our short string linker by default. This removes linking support for Arabic.
    • Additional Language Support: The entity extractor now supports Vietnamese.

    CJK Support for Names

    • Name Translation and Similarity CJK Improvements: The /name-similarity and /name-translation endpoints now support matching and translating between Japanese-Chinese, Japanese-Korean, and Korean-Chinese. Japanese accuracy was improved significantly.

    Text Embeddings Improvement

    • Improved Accuracy for Document-level Embeddings: We made some improvements to our algorithm for calculating text embeddings across multi-word input, so you should see more accurate results for document-level vectors.

    Japanese Sentiment Analysis

    • Beta Japanese Support for /sentiment: We now return document-level and entity-level sentiment analysis results for Japanese language input.
  • NEW Release - 1.4.0

    October 27, 2016

    Syntactic Dependencies (NEW)

    • New Endpoint: We've added a syntactic dependencies endpoint that returns the parse tree of the input text as a list of labeled directed links between tokens, as well as the list of tokens in the input sentence.

    Relationship Extraction

    • Entities Linked to Wikidata: Where available, Rosette will now link entities extracted within relationships to Wikidata. You'll see this information returned as a QID in the argument ID.
    • Modality Returned: We've also added a "modality" field to Rosette's Relationship Extraction. Modality is the semantic context of the possibility or necessity of the relationship; the values can be “assertion”, “negation”, “uncertainty”, “opinion”, or “question”.

    Starter Plan (NEW)

    • New $99 API Plan: For a limited time, we’re offering a special Starter plan. $99/month gets you 40,000 Rosette API calls. Want to dive deep into Rosette but don’t need a whole 100,000 calls? This plan is for you.
  • NEW Release - 1.3.0

    September 15, 2016

    Text Embedding (NEW)

    • New Endpoint: We added a text embedding endpoint that returns a single vector of floating point numbers that represents the document or word in the semantic space. Read more and try it out!

    Sentiment Analysis

    • Additional entities: We changed the /sentiment endpoint to return the sentiment of all entities discovered by Rosette, including Person, Location, Organization, Date, Time, and more entity types.

    Entity Extraction

    • Turn off entity linking: We added an option to disable entity linking in order to improve the call speed. Add "options": {"linkEntities": "false"} to your /entities call. Rosette returns a list of the entities with a temporary ID (TID).

    Global changes

    • Concurrency header: We added the X-RosetteAPI-Concurrency header to return the number of concurrent calls allowed on your plan. If you are receiving 429 errors, Too Many Requests, then Contact us for greater concurrency.

  • NEW Release - 1.2.3

    July 21, 2016

    Global changes

    • Input genre: The genre field is available for /entities and /entities/linked to indicate the input is from social media. Specifying genre=social-media does not affect the output of the other endpoints.
      Applies to: /entities, /entities/linked, /relationships, /categories, /sentiment, /language, /morphology, /tokens, /sentences.

    Entity Linking

    Entity Extraction

    • Entity endpoints unified: We combined the /entities and /entities/linked endpoints into one endpoint, /entities. Rosette now returns the entity mentions and the entityId, if available. The entityId replaced the indocChainId.
      The output of /sentiment has not changed.

    • Entities Linked deprecated: We deprecated the /entities/linked endpoint. It is still available, but we recommend that you adapt your applications to the new /entities endpoint.

    • Additional entities: Rosette now extracts more entity types: Date, Time, Longitude and Latitude, and Distance.

    • Japanese entityId: We added support for linking entities in Japanese (jpn) text to their entityId.

    • Spanish social media: We added support for extracting entities from social-media in Spanish language documents, using the genre field.

    • Malay entities: We added support for extracting entities in Malay (msa).

    Error code

    • 409 Error: We added the 409 error code for when the binding version is out of date. If you receive this error, update your binding to the most recent version.

    Sentiment Analysis

    • Spanish support: We added support for analyzing the sentiment of Spanish language documents.
  • NEW Binding Release - Ruby and R bindings

    June 20, 2016

    Bindings

    • Ruby: We added the Ruby binding to the gray column to the right and on Github. There is a Ruby gem available as well.

    • R: We added the R binding to the gray column to the right and on Github.

    • cURL examples: We changed the shell examples in the gray column on Features and Functions to be cURL code examples.

  • NEW Release - 1.1.2

    May 10, 2016

    Entity Linking

    • Social input: We added a request field, "genre": "social-media", to speed up and improve the accuracy of linking Person, Location, Organization and Product entities in social media posts. English input only.
  • NEW Release - 0.10.3

    March 29, 2016

    Global changes

      • Language used: We added a Response Header, X-RosetteAPI-ProcessedLanguage, to return the language used by Rosette for processing the call.
        Applies to: /entities, /entities/linked, /relationships, /categories, /sentiment, /language, /morphology, /tokens, /sentences

      • requestId moved: We moved the requestId object from the JSON response body to the Response Header as X-RosetteAPI-Request-Id.
        Applies to: /entities, /entities/linked, /relationships, /categories, /sentiment, /language, /morphology, /tokens, /sentences, /name-translation, /name-similarity

      • Rosette API Key: We changed the user_key header’s name to X-RosetteAPI-Key. The user_key header is deprecated.
        Applies to: /entities, /entities/linked, /relationships, /categories, /sentiment, /language, /morphology, /tokens, /sentences, /name-translation, /name-similarity

      • unit parameter removed: We removed the optional unit request parameter. All input will be handled as a doc.
        Applies to: /entities, /entities/linked, /relationships, /sentiment, /morphology

      • Base64: We removed support for Base64 encoding. You can submit binary files as a multipart/form-data call type.
        Applies to: /entities, /entities/linked, /relationships, /categories, /sentiment, /language, /morphology, /tokens, /sentences

      Entity Extraction

      • Confidence removed: The confidence value has been removed from the response object.

      Relationship Extraction

      • Accuracy mode: We removed the optional accuracy mode. All input will be processed with the precision accuracy mode, so Rosette will return a precise list of accurate relationships.
      • Explanations removed: The explanations value has been removed from the response object.

      Categorization

      • Explanations removed: The explanations value has been removed from the response object.

      Sentiment Analysis

      • Entity sentiment: We added support for entity-level sentiment analysis. The JSON response for the /sentiment endpoint now includes two objects – document and entities. See the interactive documentation for examples of this new response.

      • Neutral result: We added a neutral label for documents and entities with a neutral sentiment.

      • Short strings: Rosette will automatically process short and long content with our proprietary algorithm for sentiment analysis.

      • Explanations removed: The explanations value has been removed from the response object.

      Morphological Analysis

      • Added language support: We added language support for Dari, Persian, Urdu, and Western Farsi for Parts-of-Speech Tags.

      • Universal POS Tags: We return Universal Parts-of-Speech Tags for all supported languages.

      • Tokens list: Rosette returns parallel lists of tokens, lemmas, compound components, parts-of-speech tags, and Han-readings. If a token does not have a lemma, compound component, POS tag, or Han-reading, or if the language is not supported, then Rosette will return “null” in that list.

      Name Translation

      • Renamed to /name-translation: To clarify the endpoint’s function, we renamed /translated-name to /name-translation. /translated-name is no longer available.

      • Removed result layer: Within the response to /name-translation and /name-similarity endpoints, we removed the result layer so the results are in the response object.
        Also applies to: /name-similarity

      • TargetScheme requires uppercase: For advanced users who would like to specify a targetScheme, the scheme must be submitted in uppercase.

      Name Matching

      • Renamed to /name-similarity: To clarify the endpoint’s function, we renamed /matched-name to /name-similarity. /matched-name is no longer available.

  • Limited Public Beta Release - 0.7.0

    November 16, 2015

    Limited public release delivering almost all of the Rosette SDK functionality via RESTful calls, with these additional capabilities:

    • Relationship Extraction - for English

    Additional Bindings:

    • Node.js

  • Beta Private Release

    July 13, 2015

    Initial private release delivering almost all of the Rosette SDK functionality via RESTful calls, with these language capabilities:

    • Text Extraction from web pages
    • Language Identification - includes high accuracy for short strings
    • Morphological Analysis - for 20+ languages
    • Entity Extraction - for 17 languages, inc. Indonesian
    • Entity Linking against Wikidata - for English, Spanish and Chinese
    • Sentiment Scoring - for English
    • Categorization - for English
    • Name Translation - between English and 11 languages
    • Name Matching - including entity type and language guessing

    Additional Bindings:

    • C#
    • Java

  • Alpha Private Release

    March 15, 2015

    Private release to a select audience delivering parts of the Rosette SDK functionality via RESTful calls, including:

    • Text Extraction from web pages
    • Language Identification
    • Morphological Analysis, (tokens, lemmas and Universal POS tags)
    • Entity Extraction
    • Entity Linking through Wikidata
    • Sentiment Scoring
    • Categorization (using QAG Taxonomy)
    • Name Translation - including language of origin guessing

    Bindings:

    • cURL
    • PHP
    • Python