org.apache.tika.parser.html.charsetdetector

Class StandardHtmlEncodingDetector

java.lang.Object

org.apache.tika.parser.html.charsetdetector.StandardHtmlEncodingDetector

All Implemented Interfaces:

java.io.Serializable, EncodingDetector

public final class StandardHtmlEncodingDetector
extends java.lang.Object
implements EncodingDetector

An encoding detector that tries to respect the spirit of the HTML spec part 12.2.3 "The input byte stream", or at least the part that is compatible with the implementation of tika.

https://html.spec.whatwg.org/multipage/parsing.html#the-input-byte-stream

If a resource was fetched over HTTP, then HTTP headers should be added to tika metadata when using detect(java.io.InputStream, org.apache.tika.metadata.Metadata), especially HttpHeaders.CONTENT_TYPE, as it may contain charset information.

This encoding detector may return null if no encoding is detected. It is meant to be used inside a CompositeDetector. For instance:

 
     EncodingDetector detector = new CompositeDetector(
         new StandardHtmlEncodingDetector(),
         new Icu4jEncodingDetector()
     );
 

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
StandardHtmlEncodingDetector()

Method Summary

Modifier and Type	Method and Description
java.nio.charset.Charset	detect(java.io.InputStream input, Metadata metadata) Detects the character encoding of the given text document, or null if the encoding of the document can not be detected.
int	getMarkLimit()
void	setMarkLimit(int markLimit) How far into the stream to read for charset detection.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StandardHtmlEncodingDetector

public StandardHtmlEncodingDetector()

Method Detail

detect

public java.nio.charset.Charset detect(java.io.InputStream input,
                                       Metadata metadata)
                                throws java.io.IOException

Description copied from interface: EncodingDetector

Detects the character encoding of the given text document, or null if the encoding of the document can not be detected.

If the document input stream is not available, then the first argument may be null. Otherwise the detector may read bytes from the start of the stream to help in encoding detection. The given stream is guaranteed to support the mark feature and the detector is expected to mark the stream before reading any bytes from it, and to reset the stream before returning. The stream must not be closed by the detector.

The given input metadata is only read, not modified, by the detector.

Specified by:

detect in interface EncodingDetector

Parameters:

input - text document input stream, or null

metadata - input metadata for the document

Returns:

detected character encoding, or null

Throws:

java.io.IOException - if the document input stream could not be read

getMarkLimit

public int getMarkLimit()

setMarkLimit

@Field
public void setMarkLimit(int markLimit)

How far into the stream to read for charset detection. Default is 8192.