This repository hosts code for converting the original ConvMixer 1 (PyTorch) models to TensorFlow. In this repo we convert these 6 ConvMixer models to TensorFlow SavedModels:
The models can also be found on TensorFlow Hub
| Model Name | Fine-tunable | Resolution | acc@1 | #params | File Size | TensorFlow Model |
|---|---|---|---|---|---|---|
| ConvMixer-1536/20 | ❌ | 224x224 | 81.37 | 51.6 x 10^6 | 184MB | github/drive/bucket |
| ConvMixer-768/32 | ❌ | 224x224 | 80.16 | 21.1 x 10^6 | 75MB | github/drive/bucket |
| ConvMixer-1024/20 | ❌ | 224x224 | 76.94 | 24.4 x 10^6 | 87MB | github/drive/bucket |
| Feature Extractor ConvMixer-1536/20 | ✅ | 224x224 | - | - | 178MB | github/drive/bucket |
| Feature Extractor ConvMixer-768/32 | ✅ | 224x224 | - | - | 75MB | github/drive/bucket |
| Feature Extractor ConvMixer-1024/20 | ✅ | 224x224 | - | - | 75MB | github/drive/bucket |
You can use the Feature Extractor models for fine-tuning on your own datasets.
ConvMixer is a simple model, proposed in the paper "Patches Are All You Need?" 2 for image classification pre-trained on ImageNet-1K. ConvMixer uses only standard convolutions to achieve the mixing steps. Despite it's simplicity it outperforms ViT and MLP-Mixer. Using the conversion.ipynb notebook, one should be able to take a model from the pre-trained models and convert that to TensorFlow and use that with TensorFlow Hub and Keras.
The original model classes and weights were converted using the onnx-tf tool 3.
This notebook demonstrates how the models were converted to TensorFlow SavedModel and uses onnx-tf 3 tool to do so. This notebook loads the ConvMixer model using timm (PyTorch Image Models) 4 modifying some of the code to allow doing so as shown in the timm_convmixer.py file in this repo.
This noteboook demonstrates how to load a ConvMixer Model varaint and run image classification using TensorFlow and TensorFlow Hub 5. This notebook will soon be updated to fetch models from tfhub.dev. These models can be used in the exact same manner with TensorFlow Keras too.
Awesome! If you want to contribute to this project, you're always welcome! See Contributing Guidelines. You can also take a look at open issues for getting more information about current or upcoming tasks.
Have any questions, doubts or want to present your opinions, views? You're always welcome. You can start discussions.
Copyright 2020 Rishit Dagli
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Footnotes
-
Official Code Implementation: https://github.com/tmp-iclr/convmixer ↩
-
Anonymous. Patches Are All You Need? 2021. openreview.net, https://openreview.net/forum?id=TVHS5Y4dNvM. ↩
-
onnx-tensorflow. Tensorflow Backend for ONNX. https://github.com/onnx/onnx-tensorflow ↩ ↩2
-
Ross Wightman, . "PyTorch Image Models." https://github.com/rwightman/pytorch-image-models. (2019). ↩
-
TensorFlow Hub. https://www.tensorflow.org/hub ↩