Pull Distribution

Navigation:  Configuration > Creating and Managing Distribution >

Pull Distribution

Previous pageReturn to chapter overviewNext page

This topic deals with the creation and management of PULL distributions in the system. As the name suggests PULL distribution involves "pulling" or extraction of data from remote servers. The major protocols used here are HTTP, SCP and FTP.

 

The time format that is used for recognizing time in the file is fetched with PULL. If this time format is left empty the system will try to auto-detect the remote time format. This is used to sort files by date for instance in raw log search.

 

 

To create a PULL distribution

 

1.   Ensure you are the PULL distribution page. Refer parent topic Creating and Managing Distribution for details.

 

2.   Click add_filter_btn.  The following fields and buttons are displayed.

 

pull_distribtion_add

 

3.   Click inside the Distribution field and select a distribution from the drop-down list. This is a suggest drop-down. Here you can select an existing distribution name (this means adding a new path to an existing distribution) or create a new one.
 

 

      distribution_dropdwn

 

4.   Enter a path in the Path field.

 

Note:   '*' groups all files into the same distribution. For example: for /var/log/*.log

/var/log/a.log  and /var/log/b.log are copied to same distribution path. '%' groups all files into different distributions for example, for /var/log/%-access.log

/var/log/www.host1-access.log and /var/log/www.host2-access.log will be copied to different distributions making it possible to store each file into different distribution path.

 

A distribution path can contain wildcards. The following table explains the behavior of wildcards in more detail.

 

Wildcard character

Behavior

Example

“*”
 

Tells OTUS to copy everything that is matched by it. Useful when copying a large or variable amount of log files from a remote server.

To easily copy rotated log files - /var/log/auth.log*, as it matches all required remote files: /var/log/auth.log,/var/log/auth.log.1,/var/log/auth.log.2, etc.

“%”

Useful when various remote paths are to be matched, but are to be stored in the system separately, as they are completely different log types.

An example path to easily copy all access log files is /var/log/%-access.log, that matches all access log files but stores them separately in the system. This path will match remote files: /var/log/host1-access.log, /var/log/host2-access.log, etc.

Using "*" and "%" in combination.

A combination of the behavior of both the wildcards.

/var/log/%-access.log* matches:  /var/log/host1-access.log, /var/log/host1-access.log.1, /var/log/host2-access.log, /var/log/host2-access.log.2. And stores this data separately for each host.

 

5.   Enter a time format in the Timeformat field. Refer the topic List of OTUS time formats for more information.

 

6.   Click inside the Indexers field and from the drop-down select one or more indexers. You can search for an indexer by entering characters or words of the name of the indexer in the empty field located on drop of the drop-down list. To remove an indexer click the "X" of the name of the indexer.

 

    index_drpdn

 

 

Note:   Indexers are optional, if none is entered only raw log files are stored on server.

 

7.   Click save_btn. The new distribution is added to the list of distributions in the table.

 

 

To modify a PULL distribution

 

1.   Double-click an editable field of a record in the table and it is enabled for editing. In the example the Path field of a record has been double-clicked.

 

    path_editing_enabled

 

2.   Edit the field as required.

 

3.   Click save_btn to save the changes. Click cancel_btn to close without saving any changes made.

 

 

 

To delete a PULL distribution

 

1.   Click to select the record of a PULL distribution to be deleted.

 

2.   Click delete_btn.  The delete confirmation dialog is displayed.

 

      pull_distrib_del_cnf

 

3.   Click Yes to delete the record.

 

Caution:   Exercise this function with care. The process cannot be undone. All data is deleted.