Training models on synthetic data — Paper link The authors explore the capability of text-to-image models, particularly the Stable Diffusion model, to learn visual representations using synthetic images. They found that, with the correct configuration, training self-supervised methods on synthetic images can rival or even surpass the performance of real images. Moreover, the researchers developed a…

We had one pre-training. What about a second pre-training? Paper link This paper challenges the conventional pretrain-then-finetune approach widely used in visual recognition tasks in computer vision. The authors propose an extra pre-pretraining phase that employs a self-supervised technique, MAE (Masked Autoencoder), to initialize the model. Contrary to the previous…

They know how to use small models to reach satisfying results — Paper link Code link CUDA kernels link The paper introduces QLoRA, an efficient finetuning technique that minimizes memory usage, enabling the finetuning of a 65-billion parameter model on a single 48GB GPU. QLoRA employs a frozen, 4-bit quantized pretrained language model and backpropagates gradients into Low Rank Adapters (LoRA). The…

Hindsight is 20/20 — Paper link The authors propose a new method called “Chain of Hindsight” to help language models learn from human feedback more effectively. Traditional methods are limited due to dependency on selected model generations favored by humans or reliance on imperfect reward functions. Chain of Hindsight transforms all types of feedback…

Speech Technologies are available for most languages now — Paper link Blogpost link Code link The Massively Multilingual Speech (MMS) project aims to significantly broaden the language coverage of speech technology, which is currently limited to about 100 languages out of over 7,000 spoken worldwide. The project utilizes a new dataset derived from publicly available religious texts and maximizes…

Interactive manipulation of images — Paper link Code link Project link This research introduces DragGAN, a new method for controlling the output of generative adversarial networks (GANs), used for generating realistic images. Traditional methods for control in GANs typically rely on manually annotated training data or prior 3D models, which often lack precision, flexibility, and…

What if we train BERT on Dark Web? — Paper link This work presents DarkBERT, a new language model trained specifically on data from the Dark Web. Recognizing the unique language differences between the Dark Web and Surface Web, the researchers devised a method to filter and compile textual data from the Dark Web to train this model, addressing…

The new age of TTS models — Project link Paper link This paper presents NaturalSpeech 2, an upgraded text-to-speech (TTS) system designed to better capture the diversity and nuances of human speech, including different speaker identities, prosodies, styles, and even singing. The new system addresses the shortcomings of existing TTS systems, such as unstable prosody, word skipping/repetition…

ImageBind: Holistic AI learning across six modalities — Blogpost link Code link Paper link Demo link ImageBind is a novel approach that enables learning of a joint embedding across six different modalities — images, text, audio, depth, thermal, and IMU data. This method asserts that training does not require all combinations of paired data, but only image-paired data…