Suppress HuggingFace logging warning: "Setting `pad_token_id` to `eos_token_id`:{eos_token_id} for open-end generation."

In HuggingFace, every time I call a pipeline() object, I get a warning:

`"Setting `pad_token_id` to `eos_token_id`:{eos_token_id} for open-end generation."

How do I suppress this warning without suppressing all logging warnings? I want other warnings, but I don't want this one.

The warning comes for any text generation task done by HuggingFace. This is explained here, and you can see the code here.

Avoid that warning by manually setting the pad_token_id (e.g., to match the tokenizer or the eos_token_id).

Set the pad_token_id in the generation_config with:

model.generation_config.pad_token_id = tokenizer.pad_token_id

Alternatively, if you only need to make a single call to generate:

When you call

model.generate(**encoded_input)

just change it to

model.generate(**encoded_input, pad_token_id=tokenizer.eos_token_id)

For a text-generation pipeline, you need to set the pad_token_id in the generator call to suppress the warning:

from transformers import pipeline

generator = pipeline('text-generation', model='gpt2')
sample = generator('test test', pad_token_id=generator.tokenizer.eos_token_id)