TokenizerPreWhitespaceSplit (Apache SystemDS 3.0.0 API)
JavaScript is disabled on your browser.
Skip navigation links
Overview
Package
Class
Use
Tree
Deprecated
Index
Help
All Classes
SEARCH:
JavaScript is disabled on your browser.
Summary: 
Nested | 
Field | 
Constr | 
Method
Detail: 
Field | 
Constr | 
Method
 
Package org.apache.sysds.runtime.transform.tokenize
Class TokenizerPreWhitespaceSplit
java.lang.Object
org.apache.sysds.runtime.transform.tokenize.TokenizerPreWhitespaceSplit
All Implemented Interfaces:
Serializable, TokenizerPre
public class TokenizerPreWhitespaceSplit
extends Object
implements TokenizerPre
See Also:
Serialized Form
Constructor Summary
Constructors 
Constructor
Description
TokenizerPreWhitespaceSplit​(List<Integer> idCols,
int tokenizeCol,
org.apache.wink.json4j.JSONObject params)
 
Method Summary
All Methods Instance Methods Concrete Methods 
Modifier and Type
Method
Description
List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token>
splitToTokens​(String text)
 
List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens>
tokenizePre​(FrameBlock in)
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
Constructor Detail
TokenizerPreWhitespaceSplit
public TokenizerPreWhitespaceSplit​(List<Integer> idCols,
int tokenizeCol,
org.apache.wink.json4j.JSONObject params)
throws org.apache.wink.json4j.JSONException
Throws:
org.apache.wink.json4j.JSONException
Method Detail
splitToTokens
public List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.Token> splitToTokens​(String text)
tokenizePre
public List<org.apache.sysds.runtime.transform.tokenize.Tokenizer.DocumentToTokens> tokenizePre​(FrameBlock in)
Specified by:
tokenizePre in interface TokenizerPre
Skip navigation links
Overview
Package
Class
Use
Tree
Deprecated
Index
Help
All Classes
JavaScript is disabled on your browser.
Summary: 
Nested | 
Field | 
Constr | 
Method
Detail: 
Field | 
Constr | 
Method
Copyright © 2022 The Apache Software Foundation. All rights reserved.