magento2: Magento 2.3.5p-1 and Elasticsearch 6.8.11 fielddata error

Preconditions (*)

Magento 2.3.5p-1
Elasticsearch 6.8.11

Steps to reproduce (*)

Install magento using composer create-project --repository-url=https://repo.magento.com/ magento/project-enterprise-edition=2.3.5-p1
install Elasticsearch 6.8.11

Expected result (*)

Catalog Product page works well with updated_at sorting

Actual result (*)

Catalog Product page throw an error [{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [updated_at] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}]

We have default sorting on the catalog by updated_at desc, if I remove this sorting all works well. Elasticsearch displays updated_at as "updated_at" : { "type" : "text", "index" : false }, So I am not sure why this is happening if updated_at is not indexed and obviously can not be set as fielddata=true.

Previous version o magento that we have was 2.3.0 (magento2ce, now we moved to the project-community-edition) with elasticsearch 5.6 and all works well with such config.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 21 (7 by maintainers)

Most upvoted comments

The actual right solution for this is to REBUILD you attributes using a different storage type (such as options). But, if you really need to keep things as they are, the solution proposed by @mlaurense is the way to go. Only that his code is not a generic approach and deals with specific hardcoded fields.

I did this to automatically deal with this:

namespace MODULE\xx\Plugin\Magento\Framework;

class Elasticsearch
{
    /**
     * @var \Magento\Framework\Search\Request\Config
     */
    private $config;

    public function __construct(
        $options = [],
        $elasticsearchClient = null,
        $fieldsMappingPreprocessors = [],
        \Magento\Framework\Search\Request\Config $config
    )
    {
        $this->config = $config;
    }

    public function beforeAddFieldsMapping(
        \Magento\Elasticsearch7\Model\Client\Elasticsearch $subject,
        array $fields,
        string $index,
        string $entityType
    ) {
        $mappings = $this->config->get();

        foreach ($mappings as $configKey => $config) {
            foreach($config['aggregations'] as $aggregationKey => $aggregation) {
                if (array_key_exists('field', $aggregation)) {
                    $field = $aggregation['field'];
                    if (array_key_exists($field, $fields) && $fields[$field]['type'] === 'text') {
                        $fields[$field]['type'] = 'keyword';
                    }
                    if (array_key_exists($field . '_value', $fields) && $fields[$field . '_value']['type'] === 'text') {
                        $fields[$field]['type'] = 'keyword';
                    }
                }
            }
        }

        return [$fields, $index, $entityType];
    }
}

david-garcia-garcia on Sep 19, 2021

@JelleGe Depending on which ES version you’re using, you need to create a “before” plugin on method addFieldsMapping, changing the ES field type to keyword. In our case we made “after” plugin for \Magento\Elasticsearch7\Model\Client\Elasticsearch, so our beforeAddFieldsMapping looks like this
public function beforeAddFieldsMapping(
        Elasticsearch $subject,
        array $fields,
        string $index,
        string $entityType
    ) {
        if (!isset($fields['is_salable'])) {
            $fields['is_salable'] = [
                'type' => 'keyword'
            ];
        }

        return [$fields, $index, $entityType];
    }
You’ll have to do this for any category or product attributes for which magento uses the wrong ES field type

I can confirm this works, however in my instance the array key was already set. So the below code was needed instead of !isset (your results may vary)

if (array_key_exists('your_attribute', $fields)) { $fields['your_attribute] = [ 'type' => 'keyword' ]; }

msyhr on Dec 21, 2020

@JelleGe Depending on which ES version you’re using, you need to create a “before” plugin on method addFieldsMapping, changing the ES field type to keyword. In our case we made “after” plugin for \Magento\Elasticsearch7\Model\Client\Elasticsearch, so our beforeAddFieldsMapping looks like this

public function beforeAddFieldsMapping(
        Elasticsearch $subject,
        array $fields,
        string $index,
        string $entityType
    ) {
        if (!isset($fields['is_salable'])) {
            $fields['is_salable'] = [
                'type' => 'keyword'
            ];
        }

        return [$fields, $index, $entityType];
    }

You’ll have to do this for any category or product attributes for which magento uses the wrong ES field type

rparsi-boxycharm on Nov 9, 2020

Just letting everyone know that I’m getting this same error with m2.3.5-p2 (enterprise/whatever the paid version is called) with ES 7.0

rparsi-boxycharm on Aug 17, 2020