Skip to content

Automatic Summary

The Automatic Summary service is a tool allowing to get extrcative summary from a document. This tools is a rest service where the API is described in API section and the configuration file is described in Configuration section.

Automatic Summary API

Automatic Summary configuration

Example of Configuration:

summarizer.json
{
    "logger": {
        "logging-level": "{{ project.loglevel }}"
    },
    "summarizer": {    
        "settings":{
            "language":"en",
            "min-percent":10,
            "max-percent":20,
            "use-cuda":false,
            "cuda-device":0,
            "model-path-or-name":"{{ project.path }}/resources/modeling/net"
        },
        "network": {
            "host":"0.0.0.0",
            "port":10008,
            "associate-environment": {
                "host":"AUTOMATIC_SUMMARY_HOST",
                "port":"AUTOMATIC_SUMMARY_PORT"
            }            
        },
        "runtime":{
            "request-max-size":100000000,
            "request-buffer-queue-size":100,
            "keep-alive":true,
            "keep-alive-timeout":5,
            "graceful-shutown-timeout":15.0,
            "request-timeout":60,
            "response-timeout":60,
            "workers":1
        }
    }
}

Automatic Summary configuration is an aggreation of network configuration, serialize configuration, runtime configuration (in field converter), logger (at top level).

Configure Automatic Summary Settings

Settings (in field summarizer) allows to configure some default behaviours of summarizer:

  • settings/language: define summarizer language [en | fr]
  • settings/min-percent: minimum summarizer size in percent
  • settings/max-percent: maximum summarizer size in percent
  • settings/use-cuda: use cuda device
  • settings/cuda-device: device number (multiple graphic cards)

Configure Automatic Summary logger

Logger is configuration at top level of json in logger field.

Example of Configuration:

logger configuration
{
    "logger": {
        "logging-level": "debug"
    }    
}

The logger fields is:

  • logging-level

It can be set to the following values:

  • debug for the debug level and developper information
  • info for the level of information
  • warning to display only warning and errors
  • error to display only error
  • critical to display only error

Configure Automatic Summary Network

Example of Configuration:

network configuration
{
    "network": {
        "host":"0.0.0.0",
        "port":8080,
        "associate-environment": {
            "host":"HOST_ENVNAME",
            "port":"PORT_ENVNAME"
        },
        "ssl":
        {
            "certificate":"path/to/certificate",
            "key":"path/to/key"
        }
    }
}

The network fields:

  • host : hostname

  • port : port of the service

  • associated-environement : default one. This field is not mandatory.

    • "host" : associated "host" environment variable
    • "port" : associated "port" environment variable

  • ssl : ssl configuration IN PRODUCTION IT IS MANDATORY TO USE CERTIFICATE AND KEY THAT ARE *NOT* SELF SIGNED

  • cert : certificate file

  • key : key file

Configure Automatic Summary runtime

Example of Configuration:

network configuration
{
    "runtime":{
        "request-max-size":100000000,
        "request-buffer-queue-size":100,
        "keep-alive":true,
        "keep-alive-timeout":5,
        "graceful-shutown-timeout":15.0,
        "request-timeout":60,
        "response-timeout":60,
        "workers":1
    }    
}

The Runtime fields:

  • request-max-size : how big a request may be (bytes)

  • request-buffer-queue-size: request streaming buffer queue size

  • request-timeout : how long a request can take to arrive (sec)

  • response-timeout : how long a response can take to process (sec)

  • keep-alive: keep-alive

  • keep-alive-timeout: how long to hold a TCP connection open (sec)

  • graceful-shutdown_timeout : how long to wait to force close non-idle connection (sec)

  • workers : number of workers for the service on a node

  • associated-environement : if one of previous field is on the associated environment variables that allows to replace the default one. This field is not mandatory.

  • request-max-size : overwrite with environement variable

  • request-buffer-queue-size: overwrite with environement variable
  • request-timeout : overwrite with environement variable
  • response-timeout : overwrite with environement variable
  • keep-alive: overwrite with environement variable
  • keep-alive-timeout: overwrite with environement variable
  • graceful-shutdown_timeout : overwrite with environement variable
  • workers : overwrite with environement variable

Automatic Summary service

To run the command type simply from tkeir directory:

python3 thot/summarizer_svc.py --config=<path to summarizer configuration file>

or if you install tkeir wheel:

tkeir-summarizer-svc.py --config=<path to summarizer configuration file>

A light client can be run through the command

python3 thot/summarizer_client.py --config=<path to summarizer configuration file> --input=<input directory> --output=<output directory> -m <minimal output summary length by block> -M <maximal output summary length by block> [-mp <minimal percentage of block length> -Mp <maximal percentage of text block>]

or if you install tkeir wheel:

tkeir-summarizer-client.py --config=<path to summarizer configuration file> --input=<input directory> --output=<output directory> -m <minimal output summary length by block> -M <maximal output summary length by block> [-mp <minimal percentage of block length> -Mp <maximal percentage of text block>]

Automatic Summary Tests

The Automatic Summary service come with unit and functional testing.

Summarizer Unit tests

Unittest allows to test Tokenizer classes only.

python3 -m unittest thot/tests/unittests/TestSummarizerConfiguration.py
python3 -m unittest thot/tests/unittests/TestSummaryExtractor.py

Summarizer Functional tests

python3 -m unittest thot/tests/functional_tests/TestSummarySvc.py