Add (Ingest) Documents

The Server Library is a quick way to load a large amount of data to process.

1. Select the Server Library tab.

   Result: The folders and files in the Server Library folder A folder in which you can place source documents. When you install Sintelix you are prompted to enter a path and name for a server library folder. The default name is 'Server Library' but you can change this. The fastest way to ingest documents is to place them in a server library folder. are displayed.

   

2. Select the folder or file you want to add to the collection.

   Click on a folder or a file to select it. Double-click on a folder to expand it.

3. Select the Include documents in sub-folders checkbox, if required.

4. Select the Email notification on completion checkbox, if required.

5. Select Submit.

   Result: Sintelix will begin ingesting the documents. Progress can be viewed under the Background Tasks tab. As each document has been ingested, it is added to the Collection and can be viewed in the Documents tab.

During document uploads your browser may freeze or become unavailable. If you want to continue accessing Sintelix, open a second browser. For example, if you are uploading documents to Sintelix on Google Chrome, run another instance of Sintelix on Firefox until the upload completes.

To upload local files:

1. Select Local Files (via upload) > Upload Documents.

2. Select the file or files you want to add to the collection. Use the Shift or Ctrl key to select multiple files.
3. Select Open. The document/documents are added to the Document list under the Add Documents pane.

You can also drag and drop files from Windows Explorer onto the Local Files tab.

The From URLs tab enables you to extract text based information directly from the Internet in three ways:

• From specific web pages, for example, http://yourwebsite.com

• From twitter posts linked to a Twitter ID, for example, @yourtwitterid

• From twitter posts linked to a Twitter hashtag, for example, #yourtwitterhashtag

To ingest text or files from a URL:

1. Select From URLs tab.
2. Enter the URL you want to ingest the text from, or enter the path to a specific file on a site. To enter multiple URLs, separate each one with a line break as follows:

3. Select Submit. The document/documents are added to the Document list under the Add Documents pane.

Ingest Structured Data

Ingesting data

To ingest emails from a mail server, this feature must be enabled. (see Configure and Secure Libraries) .

To ingest emails from a mail server:

1. Select the Mail tab.
2. Enter the details of the mail server you want to ingest emails from.
3. Select Login.

Each time you log in to a mail server, the login details you entered, (but not the password), are saved in the Login History section below the login fields. Expand this section to show the details of the five most recent logins. To populate the login fields with the details of one of the mail servers (rather than completing the fields manually), select the relevant row. Enter the password then select Login.

4. Select the folder you want to ingest emails from.

By default, emails in the subfolders will be also ingested. Deselect the Recursive check box if you do not want the subfolders to be ingested.

5. Select Submit.


The document/documents are added to the Document list under the Add Documents pane.

6. To log out of the mail server, select Log out.

You can configure Sintelix to ingest historical and future tweets from Twitter based on different criteria. For example, you may want to ingest tweets from a specific account, or tweets that mention a current event or contain a specific phrase.

You must specify the maximum number of tweets you want to ingest. The ingestion is run as a background task from the time you submit the task until the maximum number of tweets have been ingested.

If you want to ingest historical tweets, a standard Twitter account will enable you to search for tweets posted in the past 7 days. Access to more data requires a paid subscription.

To ingest tweets from Twitter and add search feature:

1. Create a Twitter developer account

2. Set up an External API configuration with your Twitter credentials

3. Ingest Tweets

Create a Twitter developer account

1. In Twitter, create a developer account, if you have not already done so.
2. Log in to Twitter as a developer, then follow the steps on the Getting Started page to create an app.
3. Go to the Keys and tokens tab, then copy the following credentials, as you will need to enter them into Sintelix:
• API key
• API secret key
• Access token
• Access token secret

Set up an External API configuration with your Twitter credentials

1. In Sintelix, open the project into which you want to ingest tweets.
2. On the Main Navigation Bar select Admin.
3. Select Data Source Definitions >  Create a Credential.
   
4. When the Create a new External API Credential window displays, from API Provider, select Twitter API v1 or select Twitter API v2.
5. Enter your Twitter credentials then select Test Credentials.
6. Do one of the following:
• If your credentials are correct, a green tick is displayed. Select Save.

• If your credentials are not correct, a red cross is displayed. Check that you have correctly copied your credentials from Twitter then enter them again.

Ingest tweets

1. On the Main Navigation Bar select Collections then either open the collection in which you want to ingest the tweets or create a new collection.
2. Select the Add Documents tab, or button, then select the External API tab.

3. From the Import Configuration dropdown list select the API configuration you created with your Twitter credentials.

4. Do one or more of the following:

Set up a background task to ingest future tweets:

5. Select Stream.

6. In the Total tweets field enter the maximum number of tweets to ingest.
7. From the Language drop-down list select the desired language of the tweets.
8. Do one of the following:
• To ingest tweets that contain a specific phrase or phrases, select Phrases then enter the phrase. To enter more than one phrase, separate them with a comma.
• To ingest tweets by a specific Twitter user/s, select Accounts then enter their user name/s. If entering more than one user name, separate them with a comma.
9. Select Submit.


A background task begins to ingest tweets into the collection. Ingestion The process of converting added files into Sintelix documents in Sintelix will stop when the total number of tweets specified has been ingested.

10. To see details about the task while it is running, select the Background Tasks tab. To stop the selection select the Delete icon .


Set up a background task to search for and ingest historical tweets:

11. Select Search.

12. In the Total tweets field enter the maximum number of tweets you want to ingest.
13. From the Language drop-down list select the language in which the tweets you want to ingest were written in.
14. Expand the Phrases section to view the search criteria fields, then enter your search criteria. You must enter at least one search criterion into one of the following fields:
• Phrases
• Hashtags
• From
• To
• Mention
  Check that the documents are added to the collection successfully.

15. Select Submit. A background task begins to ingest the tweets into the collection. Ingestion The process of converting added files into Sintelix documents in Sintelix will stop when the total number of tweets you specified have been ingested.
16. To see details about the task while it is running, select Background Tasks. To stop the task select the Delete icon beside it.



You can now explore and analyse the information in the collection A collection is a container for storing and organising ingested files and documents. Only the textual content is stored in collections, not the original files and documents. (see ).