Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design the high-level abstraction for dataset transform #12

Merged
merged 2 commits into from
Feb 19, 2021

Conversation

leonwanghui
Copy link
Contributor

@leonwanghui leonwanghui commented Feb 10, 2021

This PR is proposed to add a new high-level abstraction for dataset transform, so as to furtherly simplify the data preprocessing and postprocessing operation. Currently we only add MnistTransform, Cifar10Transform and ImageFolderTransform these three classes. Here is the simple guidelines of how to use them.

MnistTransform

Train/Eval

from tinyms.data import MnistDataset
from tinyms.vision import mnist_transform

mnist_ds = MnistDataset(data_path, shuffle=True)
mnist_ds = mnist_transform.apply_ds(mnist_ds)

Predict

import cv2
import numpy as np
from tinyms.vision import mnist_transform

img_data = np.asarray(cv2.imread(img_path, cv2.IMREAD_GRAYSCALE), dtype=np.float32)
img_data = mnist_transform(img_data)

Postprocess

from tinyms.vision import mnist_transform

# By default the strategy is `TOP1_CLASS`
mnist_transform.postprocess(input)
# Specify the strategy manually
mnist_transform.postprocess(input, strategy='TOP5_CLASS')

Cifar10Transform

Train/Eval

from tinyms.data import Cifar10Dataset
from tinyms.vision import cifar10_transform

cifar_ds = Cifar10Dataset(data_path, shuffle=True)
cifar_ds = cifar10_transform.apply_ds(cifar_ds)

Predict

import numpy as np
from PIL import Image
from tinyms.vision import cifar10_transform

img_data = np.asarray(Image.open(img_path), dtype=np.float32)
img_data = cifar10_transform(img_data)

Postprocess

from tinyms.vision import cifar_transform

# By default the strategy is `TOP1_CLASS`
cifar10_transform.postprocess(input)
# Specify the strategy manually
cifar10_transform.postprocess(input, strategy='TOP5_CLASS')

@leonwanghui leonwanghui self-assigned this Feb 10, 2021
@leonwanghui leonwanghui force-pushed the vision-trans branch 3 times, most recently from 6d150de to 6c75f0d Compare February 10, 2021 08:26
@leonwanghui leonwanghui changed the title [WIP] Design the high-level abstraction for dataset transform Design the high-level abstraction for dataset transform Feb 10, 2021
Copy link
Contributor

@lyd911 lyd911 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@leonwanghui leonwanghui merged commit 6078680 into tinyms-ai:main Feb 19, 2021
@leonwanghui leonwanghui deleted the vision-trans branch February 19, 2021 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants