pyarrow.LocalFileSystem

class pyarrow.LocalFileSystem

Bases: pyarrow.filesystem.FileSystem

__init__()

Initialize self. See help(type(self)) for accurate signature.

Methods

__init__()	Initialize self.
cat(path)	Return contents of file as a bytes object.
delete(path[, recursive])	Delete the indicated file or directory.
disk_usage(path)	Compute bytes used by all contents under indicated path in file tree.
exists(path)	Return True if path exists.
get_instance()
isdir(path)	Return True if path is a directory.
isfile(path)	Return True if path is a file.
ls(path)	Return list of file paths.
mkdir(path[, create_parents])	Create a directory.
mv(path, new_path)	Alias for FileSystem.rename.
open(path[, mode])	Open file for reading or writing.
read_parquet(path[, columns, metadata, …])	Read Parquet data from path in file system.
rename(path, new_path)	Rename file, like UNIX mv command.
rm(path[, recursive])	Alias for FileSystem.delete.
stat(path)	Information about a filesystem entry.
walk(path)	Directory tree generator, see os.walk.

Attributes

pathsep

cat(path)

Return contents of file as a bytes object.

Parameters

path (str) – File path to read content from.

Returns

contents (bytes)

delete(path, recursive=False)

Delete the indicated file or directory.

Parameters

• path (str) – Path to delete.

• recursive (bool, default False) – If True, also delete child paths for directories.

disk_usage(path)

Compute bytes used by all contents under indicated path in file tree.

Parameters

path (str) – Can be a file path or directory.

Returns

usage (int)

exists(path)

Return True if path exists.

Parameters

path (str) – Path to check.

classmethod get_instance()

isdir(path)

Return True if path is a directory.

Parameters

path (str) – Path to check.

isfile(path)

Return True if path is a file.

Parameters

path (str) – Path to check.

ls(path)

Return list of file paths.

Parameters

path (str) – Directory to list contents from.

mkdir(path, create_parents=True)

Create a directory.

Parameters

• path (str) – Path to the directory.

• create_parents (bool, default True) – If the parent directories don’t exists create them as well.

mv(path, new_path)

Alias for FileSystem.rename.

open(path, mode='rb')

Open file for reading or writing.

property pathsep

read_parquet(path, columns=None, metadata=None, schema=None, use_threads=True, use_pandas_metadata=False)

Read Parquet data from path in file system. Can read from a single file or a directory of files.

Parameters

• path (str) – Single file path or directory

• columns (List[str], optional) – Subset of columns to read.

• metadata (pyarrow.parquet.FileMetaData) – Known metadata to validate files against.

• schema (pyarrow.parquet.Schema) – Known schema to validate files against. Alternative to metadata argument.

• use_threads (bool, default True) – Perform multi-threaded column reads.

• use_pandas_metadata (bool, default False) – If True and file has custom pandas schema metadata, ensure that index columns are also loaded.

Returns

table (pyarrow.Table)

rename(path, new_path)

Rename file, like UNIX mv command.

Parameters

• path (str) – Path to alter.

• new_path (str) – Path to move to.

rm(path, recursive=False)

Alias for FileSystem.delete.

stat(path)

Information about a filesystem entry.

Returns

stat (dict)

walk(path)

Directory tree generator, see os.walk.

pyarrow.CompressedOutputStream Tables and Tensors