Scripting

最新推荐文章于 2021-10-22 20:34:43 发布

转载最新推荐文章于 2021-10-22 20:34:43 发布 · 1.4k 阅读

文章标签：

#ElasticSearch Script

ElasticSearch 专栏收录该内容

2 篇文章

订阅专栏

The scripting module allows to use scripts in order to evaluate customexpressions. For example, scripts can be used to return "script fields"as part of a search request, or can be used to evaluate a custom scorefor a query and so on.

The scripting module uses by default mvel asthe scripting language with some extensions. mvel is used since it isextremely fast and very simple to use, and in most cases, simpleexpressions are needed (for example, mathematical equations).

Additional lang plugins are provided to allow to execute scripts indifferent languages. Currently supported plugins are lang-javascriptfor JavaScript, lang-groovy for Groovy, and lang-python for Python.All places where a script parameter can be used, a lang parameter(on the same level) can be provided to define the language of thescript. The lang options are mvel, js, groovy, python, andnative.

Added in 1.2.0.

Dynamic scripting is disabled by default since version 1.2.0.

To increase security, Elasticsearch does not allow you to specify scripts with arequest. Instead, scripts must be placed in the scripts directory inside theconfiguration directory (the directory where elasticsearch.yml is). Scriptsplaced into this directory will automatically be picked up and be available tobe used. Once a script has been placed in this directory, it can be referencedby name. For example, a script called calculate-score.mvel can be referencedin a request like this:

$ tree config
config
├── elasticsearch.yml
├── logging.yml
└── scripts
    └── calculate-score.mvel

$ cat config/scripts/calculate-score.mvel
Math.log(_score * 2) + my_modifier

curl -XPOST localhost:9200/_search -d '{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "body": "foo"
        }
      },
      "functions": [
        {
          "script_score": {
            "script": "calculate-score",
            "params": {
              "my_modifier": 8
            }
          }
        }
      ]
    }
  }
}'

The name of the script is derived from the hierarchy of directories itexists under, and the file name without the lang extension. For example,a script placed under config/scripts/group1/group2/test.py will benamed group1_group2_test.

Default Scripting Languageedit

The default scripting language (assuming no lang parameter isprovided) is mvel. In order to change it set the script.default_langto the appropriate language.

Enabling dynamic scriptingedit

We recommend running Elasticsearch behind an application or proxy,which protects Elasticsearch from the outside world. If users areallowed to run dynamic scripts (even in a search request), then theyhave the same access to your box as the user that Elasticsearch isrunning as. For this reason dynamic scripting is disabled by default.

First, you should not run Elasticsearch as the root user, as this would allowa script to access or do anything on your server, without limitations. Second,you should not expose Elasticsearch directly to users, but instead have a proxyapplication inbetween. If you do intend to expose Elasticsearch directly toyour users, then you have to decide whether you trust them enough to run scriptson your box or not. If you do, you can enable dynamic scripting by adding thefollowing setting to the config/elasticsearch.yml file on every node:

script.disable_dynamic: false

While this still allows execution of named scripts provided in the config, ornative Java scripts registered through plugins, it also allows users to runarbitrary scripts via the API. Instead of sending the name of the file as thescript, the body of the script can be sent instead.

Automatic Script Reloadingedit

The config/scripts directory is scanned periodically for changes.New and changed scripts are reloaded and deleted script are removedfrom preloaded scripts cache. The reload frequency can be specifiedusing watcher.interval setting, which defaults to 60s.To disable script reloading completely set script.auto_reload_enabledto false.

Native (Java) Scriptsedit

Even though mvel is pretty fast, this allows to register native Java basedscripts for faster execution.

In order to allow for scripts, the NativeScriptFactory needs to beimplemented that constructs the script that will be executed. There aretwo main types, one that extends AbstractExecutableScript and one thatextends AbstractSearchScript (probably the one most users will extend,with additional helper classes in AbstractLongSearchScript,AbstractDoubleSearchScript, and AbstractFloatSearchScript).

Registering them can either be done by settings, for example:script.native.my.type set to sample.MyNativeScriptFactory willregister a script named my. Another option is in a plugin, accessScriptModule and call registerScript on it.

Executing the script is done by specifying the lang as native, andthe name of the script as the script.

Note, the scripts need to be in the classpath of elasticsearch. Onesimple way to do it is to create a directory under plugins (choose adescriptive name), and place the jar / classes files there, they will beautomatically loaded.

Scoreedit

In all scripts that can be used in facets, allow to access the currentdoc score using doc.score.

Computing scores based on terms in scriptsedit

see advanced scripting documentation

Document Fieldsedit

Most scripting revolve around the use of specific document fields data.The doc['field_name'] can be used to access specific field data withina document (the document in question is usually derived by the contextthe script is used). Document fields are very fast to access since theyend up being loaded into memory (all the relevant field values/tokensare loaded to memory).

The following data can be extracted from a field:

Expression	Description
`doc['field_name'].value`	The native value of the field. For example,if its a short type, it will be short.
`doc['field_name'].values`	The native array values of the field. Forexample, if its a short type, it will be short[]. Remember, a field canhave several values within a single doc. Returns an empty array if thefield has no values.
`doc['field_name'].empty`	A boolean indicating if the field has novalues within the doc.
`doc['field_name'].multiValued`	A boolean indicating that the fieldhas several values within the corpus.
`doc['field_name'].lat`	The latitude of a geo point type.
`doc['field_name'].lon`	The longitude of a geo point type.
`doc['field_name'].lats`	The latitudes of a geo point type.
`doc['field_name'].lons`	The longitudes of a geo point type.
`doc['field_name'].distance(lat, lon)`	The `plane` distance (in meters)of this geo point field from the provided lat/lon.
`doc['field_name'].distanceWithDefault(lat, lon, default)`	The `plane` distance (in meters)of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].distanceInMiles(lat, lon)`	The `plane` distance (inmiles) of this geo point field from the provided lat/lon.
`doc['field_name'].distanceInMilesWithDefault(lat, lon, default)`	The `plane` distance (inmiles) of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].distanceInKm(lat, lon)`	The `plane` distance (inkm) of this geo point field from the provided lat/lon.
`doc['field_name'].distanceInKmWithDefault(lat, lon, default)`	The `plane` distance (inkm) of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].arcDistance(lat, lon)`	The `arc` distance (inmeters) of this geo point field from the provided lat/lon.
`doc['field_name'].arcDistanceWithDefault(lat, lon, default)`	The `arc` distance (inmeters) of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].arcDistanceInMiles(lat, lon)`	The `arc` distance (inmiles) of this geo point field from the provided lat/lon.
`doc['field_name'].arcDistanceInMilesWithDefault(lat, lon, default)`	The `arc` distance (inmiles) of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].arcDistanceInKm(lat, lon)`	The `arc` distance (inkm) of this geo point field from the provided lat/lon.
`doc['field_name'].arcDistanceInKmWithDefault(lat, lon, default)`	The `arc` distance (inkm) of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].factorDistance(lat, lon)`	The distance factor of this geo point field from the provided lat/lon.
`doc['field_name'].factorDistance(lat, lon, default)`	The distance factor of this geo point field from the provided lat/lon with a default value.
`doc['field_name'].geohashDistance(geohash)`	The `arc` distance (in meters)of this geo point field from the provided geohash.
`doc['field_name'].geohashDistanceInKm(geohash)`	The `arc` distance (in km)of this geo point field from the provided geohash.
`doc['field_name'].geohashDistanceInMiles(geohash)`	The `arc` distance (inmiles) of this geo point field from the provided geohash.

Stored Fieldsedit

Stored fields can also be accessed when executing a script. Note, theyare much slower to access compared with document fields, as they are notloaded into memory. They can be simply accessed using_fields['my_field_name'].value or _fields['my_field_name'].values.

Source Fieldedit

The source field can also be accessed when executing a script. Thesource field is loaded per doc, parsed, and then provided to the scriptfor evaluation. The _source forms the context under which the sourcefield can be accessed, for example _source.obj2.obj1.field3.

Accessing _source is much slower compared to using _docbut the data is not loaded into memory. For a single field access _fields may befaster than using _source due to the extra overhead of potentially parsing large documents.However, _source may be faster if you access multiple fields or if the source has already beenloaded for other purposes.

mvel Built In Functionsedit

There are several built in functions that can be used within scripts.They include:

Function	Description
`time()`	The current time in milliseconds.
`sin(a)`	Returns the trigonometric sine of an angle.
`cos(a)`	Returns the trigonometric cosine of an angle.
`tan(a)`	Returns the trigonometric tangent of an angle.
`asin(a)`	Returns the arc sine of a value.
`acos(a)`	Returns the arc cosine of a value.
`atan(a)`	Returns the arc tangent of a value.
`toRadians(angdeg)`	Converts an angle measured in degrees to anapproximately equivalent angle measured in radians
`toDegrees(angrad)`	Converts an angle measured in radians to anapproximately equivalent angle measured in degrees.
`exp(a)`	Returns Euler’s number e raised to the power of value.
`log(a)`	Returns the natural logarithm (base e) of a value.
`log10(a)`	Returns the base 10 logarithm of a value.
`sqrt(a)`	Returns the correctly rounded positive square root of avalue.
`cbrt(a)`	Returns the cube root of a double value.
`IEEEremainder(f1, f2)`	Computes the remainder operation on twoarguments as prescribed by the IEEE 754 standard.
`ceil(a)`	Returns the smallest (closest to negative infinity) valuethat is greater than or equal to the argument and is equal to amathematical integer.
`floor(a)`	Returns the largest (closest to positive infinity) valuethat is less than or equal to the argument and is equal to amathematical integer.
`rint(a)`	Returns the value that is closest in value to the argumentand is equal to a mathematical integer.
`atan2(y, x)`	Returns the angle theta from the conversion ofrectangular coordinates (x, y) to polar coordinates (r,theta).
`pow(a, b)`	Returns the value of the first argument raised to thepower of the second argument.
`round(a)`	Returns the closest int to the argument.
`random()`	Returns a random double value.
`abs(a)`	Returns the absolute value of a value.
`max(a, b)`	Returns the greater of two values.
`min(a, b)`	Returns the smaller of two values.
`ulp(d)`	Returns the size of an ulp of the argument.
`signum(d)`	Returns the signum function of the argument.
`sinh(x)`	Returns the hyperbolic sine of a value.
`cosh(x)`	Returns the hyperbolic cosine of a value.
`tanh(x)`	Returns the hyperbolic tangent of a value.
`hypot(x, y)`	Returns sqrt(x2 + y2) without intermediate overflowor underflow.

Arithmetic precision in MVELedit

When dividing two numbers using MVEL based scripts, the engine tries tobe smart and adheres to the default behaviour of java. This means if youdivide two integers (you might have configured the fields as integer inthe mapping), the result will also be an integer. This means, if acalculation like 1/num is happening in your scripts and num is aninteger with the value of 8, the result is 0 even though you wereexpecting it to be 0.125. You may need to enforce precision byexplicitly using a double like 1.0/num in order to get the expectedresult.