Open in app

Sign In

Write

Sign In

Andrew Lukyanenko
Andrew Lukyanenko

1.5K Followers

Home

About

Published in

Towards Data Science

·Pinned

A long-term Data Science roadmap which WON’T help you become an expert in only several months

Some thoughts on becoming a data scientist. It isn’t easy or fast and requires a lot of efforts, but if you are interested in data science, it is worth it. — From time to time I am asked: how does one become a data scientist? What courses are necessary? How long will it take? How did you become a DS? …

Data Science

7 min read

A long-term Data Science roadmap which WON’T help you become an expert in only several months
A long-term Data Science roadmap which WON’T help you become an expert in only several months
Data Science

7 min read


21 hours ago

Paper Review: StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners

Training models on synthetic data — Paper link The authors explore the capability of text-to-image models, particularly the Stable Diffusion model, to learn visual representations using synthetic images. They found that, with the correct configuration, training self-supervised methods on synthetic images can rival or even surpass the performance of real images. Moreover, the researchers developed a…

Paper Review

6 min read

Paper Review: StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual…
Paper Review: StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual…
Paper Review

6 min read


3 days ago

Paper Review: The effectiveness of MAE pre-pretraining for billion-scale pertaining

We had one pre-training. What about a second pre-training? Paper link This paper challenges the conventional pretrain-then-finetune approach widely used in visual recognition tasks in computer vision. The authors propose an extra pre-pretraining phase that employs a self-supervised technique, MAE (Masked Autoencoder), to initialize the model. Contrary to the previous…

Paper Review

4 min read

Paper Review: The effectiveness of MAE pre-pretraining for billion-scale pertaining
Paper Review: The effectiveness of MAE pre-pretraining for billion-scale pertaining
Paper Review

4 min read


Published in

GoPenAI

·Jun 1

Paper Review: QLoRA: Efficient Finetuning of Quantized LLMs

They know how to use small models to reach satisfying results — Paper link Code link CUDA kernels link The paper introduces QLoRA, an efficient finetuning technique that minimizes memory usage, enabling the finetuning of a 65-billion parameter model on a single 48GB GPU. QLoRA employs a frozen, 4-bit quantized pretrained language model and backpropagates gradients into Low Rank Adapters (LoRA). The…

Paper Review

7 min read

Paper Review: QLoRA: Efficient Finetuning of Quantized LLMs
Paper Review: QLoRA: Efficient Finetuning of Quantized LLMs
Paper Review

7 min read


May 30

Paper Review: Chain of Hindsight Aligns Language Models with Feedback

Hindsight is 20/20 — Paper link The authors propose a new method called “Chain of Hindsight” to help language models learn from human feedback more effectively. Traditional methods are limited due to dependency on selected model generations favored by humans or reliance on imperfect reward functions. Chain of Hindsight transforms all types of feedback…

Paper Review

5 min read

Paper Review: Chain of Hindsight Aligns Language Models with Feedback
Paper Review: Chain of Hindsight Aligns Language Models with Feedback
Paper Review

5 min read


May 25

Paper Review: MMS: Scaling Speech Technology to 1000+ languages

Speech Technologies are available for most languages now — Paper link Blogpost link Code link The Massively Multilingual Speech (MMS) project aims to significantly broaden the language coverage of speech technology, which is currently limited to about 100 languages out of over 7,000 spoken worldwide. The project utilizes a new dataset derived from publicly available religious texts and maximizes…

Paper Review

13 min read

Paper Review: MMS: Scaling Speech Technology to 1000+ languages
Paper Review: MMS: Scaling Speech Technology to 1000+ languages
Paper Review

13 min read


May 22

Paper Review: Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold

Interactive manipulation of images — Paper link Code link Project link This research introduces DragGAN, a new method for controlling the output of generative adversarial networks (GANs), used for generating realistic images. Traditional methods for control in GANs typically rely on manually annotated training data or prior 3D models, which often lack precision, flexibility, and…

Paper Review

6 min read

Paper Review: Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
Paper Review: Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
Paper Review

6 min read


May 18

Paper Review: DarkBERT: A Language Model for the Dark Side of the Internet

What if we train BERT on Dark Web? — Paper link This work presents DarkBERT, a new language model trained specifically on data from the Dark Web. Recognizing the unique language differences between the Dark Web and Surface Web, the researchers devised a method to filter and compile textual data from the Dark Web to train this model, addressing…

Paper Review

7 min read

Paper Review: DarkBERT: A Language Model for the Dark Side of the Internet
Paper Review: DarkBERT: A Language Model for the Dark Side of the Internet
Paper Review

7 min read


May 15

Paper Review: NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers

The new age of TTS models — Project link Paper link This paper presents NaturalSpeech 2, an upgraded text-to-speech (TTS) system designed to better capture the diversity and nuances of human speech, including different speaker identities, prosodies, styles, and even singing. The new system addresses the shortcomings of existing TTS systems, such as unstable prosody, word skipping/repetition…

Paper Review

8 min read

Paper Review: NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing…
Paper Review: NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing…
Paper Review

8 min read


May 10

Paper Review: ImageBind: One Embedding Space To Bind Them All

ImageBind: Holistic AI learning across six modalities — Blogpost link Code link Paper link Demo link ImageBind is a novel approach that enables learning of a joint embedding across six different modalities — images, text, audio, depth, thermal, and IMU data. This method asserts that training does not require all combinations of paired data, but only image-paired data…

Paper Review

8 min read

Paper Review: ImageBind: One Embedding Space To Bind Them All
Paper Review: ImageBind: One Embedding Space To Bind Them All
Paper Review

8 min read

Andrew Lukyanenko

Andrew Lukyanenko

1.5K Followers

Russian by birth. Economist by education. Polyglot as a hobby. DS as a calling. https://andlukyane.com/

Following
  • Martin Heinz

    Martin Heinz

  • Denis Vorotyntsev

    Denis Vorotyntsev

  • Vishnu Subramanian

    Vishnu Subramanian

  • Mark Andreev

    Mark Andreev

  • Ivan Panshin

    Ivan Panshin

See all (33)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech

Teams