Package ca.phon.ipadictionary.impl

Class TransliterationTokenizer

java.lang.Object
ca.phon.ipadictionary.impl.TransliterationTokenizer
public class TransliterationTokenizer extends Object
Create tokens from a given input string. The string will be tokenized based on a given input map or by each character if a sequence is not found in the map.

  • Constructor Details

    • TransliterationTokenizer

      public TransliterationTokenizer()

    • TransliterationTokenizer

      public TransliterationTokenizer(Map<String,String> tokenMap)

  • Method Details

    • getTokenMap

      public Map<String,String> getTokenMap()

    • isIgnoreWhitespace

      public boolean isIgnoreWhitespace()

    • setIgnoreWhitespace

      public void setIgnoreWhitespace(boolean ignoreWhitespace)

    • isEmpty

      public boolean isEmpty()

    • containsKey

      public boolean containsKey(String key)

    • get

      public String get(String key)

    • put

      public String put(String key, String value)

    • keySet

      public Set<String> keySet()

    • tokenize

      public String[] tokenize(CharSequence input)

    • transliterate

      public String transliterate(String input)

    • transliterate

      public String transliterate(String[] tokens)