Mingxing Tan
Skip to main content
Explore our many areas of focus
Explore all research areas
Applied AI & sciences
Earth AI
Health AI
Science AI
Sustainability & crisis resilience
Foundational ML & algorithms
Algorithms & theory
Information retrieval
Machine intelligence
Machine perception
Natural language processing
People, systems & quantum AI
Human-computer interaction and visualization
Networking
Quantum AI
Responsible AI
Anti abuse
Software engineering
Software systems
Learn More
Publications
Projects
Building a collaborative ecosystem
Datasets
Access high-quality datasets to accelerate your research.
Tools & services
Explore our latest AI models and products.
Open source
Discover open-source code and collaborate with the community.
Shaping the future together
See all programs
Faculty programs
Participating in the academic research community through meaningful engagement with university faculty.
Student programs
Supporting the next generation of researchers through a wide range of programming.
Locations
Find your place in our global offices and research labs.
Translating discovery into real-world impact
People
Our researchers drive advancements in computer science through both fundamental and applied research.
Teams
Collaborative groups tackling the world's most challenging AI problems.
Research
Explore our many areas of focus
Explore all research areas
Applied AI & sciences
Earth AI
Health AI
Science AI
Sustainability & crisis resilience
Foundational ML & algorithms
Algorithms & theory
Information retrieval
Machine intelligence
Machine perception
Natural language processing
People, systems & quantum AI
Human-computer interaction and visualization
Networking
Quantum AI
Responsible AI
Anti abuse
Software engineering
Software systems
Learn More
Publications
Projects
Resources
Building a collaborative ecosystem
Datasets
Access high-quality datasets to accelerate your research.
Tools & services
Explore our latest AI models and products.
Open source
Discover open-source code and collaborate with the community.
Conferences & events
Careers
Shaping the future together
See all programs
Faculty programs
Participating in the academic research community through meaningful engagement with university faculty.
Student programs
Supporting the next generation of researchers through a wide range of programming.
Locations
Find your place in our global offices and research labs.
Blog
About
Translating discovery into real-world impact
People
Our researchers drive advancements in computer science through both fundamental and applied research.
Teams
Collaborative groups tackling the world's most challenging AI problems.
Google
Research
Google AI
Learn about all our AI
Google DeepMind
Explore the frontier of AI
Google Labs
Try our AI experiments
Research
Resources
Conferences & events
Careers
Blog
About
Search
Home
People
Mingxing Tan
Research Areas
Machine intelligence
Machine perception
Authored Publications
See Filters
results
Filter by:
Clear
Publications
Google
14
Other
0
Years
2021
4
2020
7
2019
3
Research Areas
Hardware and Architecture
1
Machine Intelligence
12
Machine Perception
11
Teams
Sort By
Title
Title, descending
Year
Year, descending
chip template
Remove
MoViNets: Mobile Video Networks for Efficient Video Recognition
Dan Kondratyuk
Liangzhe Yuan
Yandong Li
Li Zhang
Mingxing Tan
Matthew Brown
Boqing Gong
CVPR 2021 (2021)
Preview
Preview abstract
We present Mobile Video Networks (MoViNets), a family of computation and memory efficient video networks that can operate on streaming video for online inference. 3D convolutional neural networks (CNNs) are accurate at video recognition but require large computation and memory budgets and do not support online inference, making them difficult to work on mobile devices. We propose a three-step approach to improve computational efficiency while substantially reducing the peak memory usage of 3D CNNs. First, we design a video network search space and employ neural architecture search to generate efficient and diverse 3D CNN architectures. Second, we introduce the Stream Buffer technique that decouples memory from video clip duration, allowing 3D CNNs to embed arbitrary-length streaming video sequences for both training and inference with a small constant memory footprint. Third, we propose a simple ensembling technique to improve accuracy further without sacrificing efficiency. These three progressive techniques allow MoViNets to achieve state-of-the-art accuracy and efficiency on the Kinetics, Moments in Time, and Charades video action recognition datasets. For instance, MoViNet-A5-Stream achieves the same accuracy as X3D-XL on Kinetics 600 while requiring 80% fewer FLOPs and 65% less memory. Code is available at https://github.com/tensorflow/models/tree/master/official/projects/movinet.
View details
AutoHAS: Efficient Hyperparameter and Architecture Search
Xuanyi Dong
Mingxing Tan
Adams Wei Yu
Daiyi Peng
Bogdan Gabrys
Quoc V. Le
Workshop on Neural Architecture Search at International Conference on Learning Representations (NAS@ICLR) (2021)
Preview
Preview abstract
Efficient hyperparameter or architecture search methods have shown remarkable results, but each of them is only applicable to searching for either hyperparameters (HPs) or architectures. In this work, we propose a unified pipeline, AutoHAS, to efficiently search for both architectures and hyperparameters. AutoHAS learns to alternately update the shared network weights and a reinforcement learning (RL) controller, which learns the probability distribution for the architecture candidates and HP candidates. A temporary weight is introduced to store the updated weight from the selected HPs (by the controller), and a validation accuracy based on this temporary weight serves as a reward to update the controller. In experiments, we show AutoHAS is efficient and generalizable to different search spaces, baselines and datasets. In particular, AutoHAS can improve the accuracy over popular network architectures, such as ResNet and EfficientNet, on CIFAR-10/100, ImageNet, and four more other datasets.
View details
EfficientNetV2: Smaller Models and Faster Training
Mingxing Tan
Quoc V. Le
International Conference on Machine Learning (ICML) (2021)
Preview
Preview abstract
This paper introduces EfficientNetV2, a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models. To develop these models, we use a combination of training-aware neural architecture search and scaling, to jointly optimize training speed and parameter efficiency. The models were searched from the search space enriched with new ops such as Fused-MBConv. Our experiments show that EfficientNetV2 models train much faster than state-of-the-art models while being up to 6.8x smaller.
Our training can be further sped up by progressively increasing the image size during training, but it often causes a drop in accuracy. To compensate for this accuracy drop, we propose an improved method of progressive learning, which adaptively adjusts regularization (e.g. data augmentation) along with image size. With progressive learning, our EfficientNetV2 significantly outperforms previous models on ImageNet and CIFAR/Cars/Flowers datasets. By pretraining on the same ImageNet21k, our EfficientNetV2 achieves 87.3% top-1 accuracy on ImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy while training 5x-11x faster using the same computing resources. Code is available at https://github.com/google/automl/tree/master/efficientnetv2.
View details
Searching for Fast Models on Datacenter Accelerators
Sheng Li
Mingxing Tan
Ruoming Pang
Andrew Li
Liqun (Legion) Cheng
Quoc V. Le
Norm Jouppi
Conference on Computer Vision and Pattern Recognition (2021)
Preview
Preview abstract
Neural Architecture Search (NAS), together with model scaling, has shown remarkable progress in designing high accuracy and fast convolutional architecture families. However, as neither NAS nor model scaling considers sufficient hardware architecture details, they do not take full advantage of the emerging datacenter (DC) accelerators. In this paper, we search for fast and accurate CNN model families
for efficient inference on DC accelerators. We first analyze DC accelerators and find that existing CNNs suffer from insufficient operational intensity, parallelism, and execution efficiency and exhibit FLOPs-latency nonproportionality. These insights let us create a DC-accelerator-optimized search space, with space-to-depth, space-to-batch, hybrid fused convolution structures with vanilla and depthwise convolutions, and block-wise activation functions. We further propose a latency-aware compound scaling (LACS), the first multi-objective compound scaling method optimizing both accuracy and latency. Our LACS discovers that network depth should grow much faster than image size and network width, which is quite different from the observations from previous compound scaling. With the new search space
and LACS, our search and scaling on datacenter accelerators results in a new model series named EfficientNet-X. EfficientNet-X is up to more than 2X faster than EfficientNet (a model series with state-of-the-art trade-off on FLOPs and accuracy) on TPUv3 and GPUv100, with comparable accuracy. EfficientNet-X is also up to 7X faster than recent RegNet and ResNeSt on TPUv3 and GPUv100. Source code is at https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/tpu
View details
Adversarial Examples Improve Image Recognition
Cihang Xie
Mingxing Tan
Boqing Gong
Jiang Wang
Alan Yuille
Quoc V. Le
CVPR (2020)
Preview
Preview abstract
Adversarial examples are commonly viewed as a threat to ConvNets. Here we present an opposite perspective: adversarial examples can be used to improve image recognition models if harnessed in the right manner. We propose AdvProp, an enhanced adversarial training scheme which treats adversarial examples as additional examples, to prevent overfitting. Key to our method is the usage of a separate auxiliary batch norm for adversarial examples, as they have different underlying distributions to normal examples.
We show that AdvProp improves a wide range of models on various image recognition tasks and performs better when the models are bigger. For instance, by applying AdvProp to the latest EfficientNet-B7 [28] on ImageNet, we achieve significant improvements on ImageNet (+0.7%), ImageNet-C (+6.5%), ImageNet-A (+7.0%), Stylized-ImageNet (+4.8%). With an enhanced EfficientNet-B8, our method achieves the state-of-the-art 85.5% ImageNet top-1 accuracy without extra data. This result even surpasses the best model in [20] which is trained with 3.5B Instagram images (~3000X more than ImageNet) and ~9.4X more parameters
View details
When Ensembling Smaller Models is More Efficient than Single Large Models
Dan Kondratyuk
Mingxing Tan
Matthew Brown
Boqing Gong
Visual Understanding by Learning from Web Data 2020, CVPR (2020)
Preview
Preview abstract
Ensembling is a simple and popular technique for boosting evaluation performance by training multiple models (e.g., with different initializations) and aggregating their predictions. This approach is commonly reserved for the largest models, as it is commonly held that increasing the model size provides a more substantial reduction in error than ensembling smaller models. However, we show results from experiments on CIFAR-10 and ImageNet that ensembles can outperform single models with both higher accuracy and requiring fewer total FLOPs to compute, even when those individual models' weights and hyperparameters are highly optimized. Furthermore, this gap in improvement widens as models become large. This presents an interesting observation that output diversity in ensembling can often be more efficient than training larger models, especially when the models approach the size of what their dataset can foster. Instead of using the common practice of tuning a single large model, one can use ensembles as a more flexible trade-off between a model's inference speed and accuracy. This also potentially eases hardware design, e.g., an easier way to parallelize the model across multiple workers for real-time or distributed inference.
View details
SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization
Xianzhi Du
Tsung-Yi Lin
Pengchong Jin
Golnaz Ghiasi
Mingxing Tan
Yin Cui
Quoc Le
Xiaodan Song
CVPR (2020)
Preview
Preview abstract
Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection). The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by ~3% AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.5% AP with a MaskR-CNN detector and achieves 52.1% AP with a RetinaNet detector on COCO for a single model without test-time augmentation, significantly outperforms prior art of detectors. SpineNet can transfer to classification tasks, achieving 5% top-1 accuracy improvement on a challenging iNaturalist fine-grained dataset. Code is at: https://github.com/tensorflow/tpu/tree/master/models/official/detection.
View details
BigNAS: Scaling Up Neural Architecture Search with Big Single-Stage Models
Jiahui Yu
Pengchong Jin
Hanxiao Liu
Gabriel M. Bender
Pieter-Jan Kindermans
Mingxing Tan
Thomas Huang
Xiaodan Song
Ruoming Pang
Quoc V. Le
European Conference on Computer Vision (ECCV) (2020)
Preview
Preview abstract
Neural architecture search (NAS) has shown promising results discovering models that are both accurate and fast. For NAS, training a one-shot model has become a popular strategy to rank the relative quality of different architectures (child models) using a single set of shared weights. However, while one-shot model weights can effectively rank different network architectures, the absolute accuracies from these shared weights are typically far below those obtained from stand-alone training. To compensate, existing methods assume that the weights must be retrained, finetuned, or otherwise post-processed after the search is completed. These steps significantly increase the compute requirements and complexity of the architecture search and model deployment. In this work, we propose BigNAS, an approach that challenges the conventional wisdom that post-processing of the weights is necessary to get good prediction accuracies. Without extra retraining or post-processing steps, we are able to train a single set of shared weights on ImageNet and use these weights to obtain child models whose sizes range from 200 to 1000 MFLOPs. Our discovered model family, BigNASModels, achieve top-1 accuracies ranging from 76.5% to 80.9%, surpassing state-of-the-art models in this range including EfficientNets and Once-for-All networks without extra retraining or post-processing. We present ablative study and analysis to further understand the proposed BigNASModels.
View details
PyGlove: Symbolic Programming for Automated Machine Learning
Daiyi Peng
Xuanyi Dong
Esteban Real
Mingxing Tan
Yifeng Lu
Hanxiao Liu
Gabriel Bender
Adam Kraft
Chen Liang
Quoc V. Le
Neural Information Processing Systems (NeurIPS) (2020)
Preview
Preview abstract
Neural networks are sensitive to hyper-parameter and architecture choices. Automated Machine Learning (AutoML) is a promising paradigm for automating these choices. Current ML software libraries, however, are quite limited in handling the dynamic interactions among the components of AutoML. For example, efficientNAS algorithms, such as ENAS and DARTS, typically require an implementation coupling between the search space and search algorithm, the two key components in AutoML. Furthermore, implementing a complex search flow, such as searching architectures within a loop of searching hardware configurations, is difficult. To summarize, changing the search space, search algorithm, or search flow in current ML libraries usually requires a significant change in the program logic. In this paper, we introduce a new way of programming AutoML based on symbolic programming. Under this paradigm, ML programs are mutable, thus can be manipulated easily by another program. As a result, AutoML can be reformulated as an automated process of symbolic manipulation. With this formulation, we decouple the triangle of the search algorithm, the search space and the child program. This decoupling makes it easy to change the search space and search algorithm (without and with weight sharing), as well as to add search capabilities to existing code and implement complex search flows. We then introduce PyGlove, a new Python library that implements this paradigm. Through case studies on ImageNet and NAS-Bench-101, we show that with PyGlove users can easily convert a static program into a search space, quickly iterate on the search spaces and search algorithms, and craft complex search flows to achieve better results.
View details
EfficientDet: Scalable and Efficient Object Detection
Mingxing Tan
Ruoming Pang
Quoc V. Le
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2020)
Preview
Preview abstract
Model efficiency has become increasingly important in computer vision. In this paper, we systematically study neural network architecture design choices for object detection and propose several key optimizations to improve efficiency. First, we propose a weighted bi-directional feature pyramid network (BiFPN), which allows easy and fast multi-scale feature fusion; Second, we propose a compound scaling method that uniformly scales the resolution, depth, and width for all backbone, feature network, and box/class prediction networks at the same time. Based on these optimizations and better backbones, we have developed a new family of object detectors, called EfficientDet, which consistently achieve much better efficiency than prior art across a wide spectrum of resource constraints. In particular, with single-model and single-scale, our EfficientDet-D7 achieves state-of-the-art 53.7 AP on COCO test-dev with 52M parameters and 325B FLOPs, being 4x - 9x smaller and using 13x - 42x fewer FLOPs than previous detectors. Code is available at https://github.com/google/automl/tree/master/efficientdet.
View details
1
2
of 2
of 2 pages
Search on Google Scholar
Join us
We're always looking for more talented, passionate people.
See opportunities
Follow us
Explore our other initiatives
Google AI
Discover how Google AI is committed to enriching knowledge and solving complex challenges
Products
Build
Research
Responsibility
Societal Impact
About
Google Cloud
High-performance infrastructure for cloud computing, data analytics & machine learning
Overview
Solutions
Products
Pricing
Resources
Google DeepMind
Our mission is to build AI responsibly to benefit humanity
Models
Research
Science
About
Google Labs
Explore the future of AI responsibly with Google Labs
About
Experiments
Stay connected
About Google
Google Products
Privacy
Terms
Cookies management controls
×