Logo

petastorm

https://secure.travis-ci.org/uber/petastorm.svg?branch=master

Navigation

  • User guide
  • How to Contribute
  • API
  • Developer guide
  • Benchmarks
  • Troubleshooting
  • Release notes
  • Migration notes

Related Topics

  • Documentation overview
    • Next: User guide

Quick search

Petastorm¶

Petastorm is a library enabling the use of Parquet storage from Tensorflow, Pytorch, and other Python-based ML training frameworks.

Github project page

  • User guide
    • Installation
    • Generating a dataset
    • Plain Python API
    • Tensorflow API
    • Pytorch API
    • Spark Dataset Converter API
    • Analyzing petastorm datasets using PySpark and SQL
    • Non Petastorm Parquet Stores
    • Troubleshooting
    • See also
  • How to Contribute
  • API
    • Unischema
    • Reader
    • Tensorflow
    • PyTorch
    • PySpark Dataset Converter
    • PySpark & SQL
    • TransformSpec
    • Row queries
    • Local cache
    • Codecs
    • Dataset generation
    • Row-group selectors
    • Benchmarks
    • HDFS
  • Developer guide
    • Setting up development environment (recommendation)
    • Unit tests
    • Documentation
    • Release procedure
  • Benchmarks
  • Troubleshooting
    • Trouble running the pytorch example
  • Release notes
    • Release 0.11.6 (unreleased)
    • Release 0.11.5
    • Release 0.11.4
    • Release 0.11.3
    • Release 0.11.2
    • Release 0.11.1
    • Release 0.11.0
    • Release 0.10.0
    • Release 0.9.8
    • Release 0.9.7
    • Release 0.9.6
    • Release 0.9.5
    • Release 0.9.4
    • Release 0.9.3
    • Release 0.9.2
    • Release 0.9.1
    • Release 0.9.0
    • Release 0.8.2
    • Release 0.8.1
    • Release 0.8.0
    • Release 0.7.7
    • Release 0.7.6
    • Release 0.7.5
    • Release 0.7.4
    • Release 0.7.3
    • Release 0.7.2
    • Release 0.7.1
    • Release 0.7.0
    • Release 0.6.0
    • Release 0.5.1
    • Release 0.5.0
    • Release 0.4.3
    • Release 0.4.2
    • Release 0.4.0, 0.4.1
  • Migration notes
    • To petastorm 0.5.0

Indices and tables¶

  • Index
  • Module Index
  • Search Page
©2017-2022, Uber Technologies, Inc.. | Powered by Sphinx 1.8.6 & Alabaster 0.7.12 | Page source