Skin Cancer Detection with ViTs
Date:
Skin cancer is one of the most common types of cancer worldwide, with millions of new cases diagnosed each year. Early detection is critical in improving patient outcomes, as it increases the chances of successful treatment. In recent years, the advancement of artificial intelligence (AI) techniques has opened new possibilities for detecting skin cancer at an early stage using automated methods. Specifically, Vision Transformers (ViTs) have emerged as a powerful tool in the field of medical imaging, providing state-of-the-art performance in tasks like image classification and segmentation. 
The goal of this project is to explore the use of ViTs for the classification of skin cancer images. Using data from the ISIC 2024 Challenge, this project investigates the efficacy of ViTs in identifying high-risk cancerous lesions from high-resolution 3D Total Body Photography (3D-TBP) images. By comparing multiple variants of ViT models, this study aims to determine which architecture offers the best performance for skin cancer detection. The dataset used in this project is provided by the ISIC 2024 Challenge, which focuses on detecting skin cancer from high-resolution 3D-TBP images. These images pose a challenge due to their high resolution, requiring efficient model architectures that can handle large amounts of data while maintaining precision in prediction. The LeViT model achieved the highest accuracy (94%), while models like CaiT and Simple ViT also performed well. Token-to-Token ViT ran into memory issues, likely due to its high-resolution patching technique.
.png?generation=1723649425473443&alt=media)
| Model | real score | Is it Ploto? | Best Acc | Number of Epochs | Number of para | paper |
|---|---|---|---|---|---|---|
| Simple ViT | 101 | yes | 92 | 50 | 53533698 | https://arxiv.org/abs/2205.01580 |
| CaiT | 97 | yes | 93 | 10 | 120801282 | https://arxiv.org/abs/2103.17239 |
| Vision Transformer for Small Datasets | 132 | yes | 92 | 50 | 54528520 | https://arxiv.org/abs/2112.13492 |
| Token to Token ViT | 113 | out of memory | 92 | 50 | 19801116 | https://arxiv.org/abs/2101.11986 |
| LeViT | 125 | yes | 94 | 100 | 17348972 | https://arxiv.org/abs/2104.01136 |
| ats ViT | 113 | yes | 92 | 25 | 52463080 | https://arxiv.org/abs/2111.15667 |
| xcit | 122 | yes | 92 | 25 | 122141864 | https://arxiv.org/abs/2106.09681 |
| vit_with_patch_merger | 118 | yes | 92 | 25 | 77687272 | https://arxiv.org/abs/2202.12015 |