[Journey] 5 x AI Sites | 5 Different Approaches | 1 Million Cumulative UVs/Month

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
How many days would it take to train GPT-J-6B model using 1080Ti GPU?

I'm planning to learn more about it. I thought it would cost more to train a model.

GPT-J-6B is a pre-trained model. It can be further fine-tuned.

In order for it to run on a 1080Ti - @Cognitive mentions the 8-bit variation. It is a quantized 8-bit version - https://huggingface.co/hivemind/gpt-j-6B-8bit - that can run on a single GPU

How many days it would take would depend on your dataset.
 

prvncqr

Jr. VIP
Jr. VIP
Joined
Jan 3, 2022
Messages
315
Reaction score
107
GPT-J-6B is a pre-trained model. It can be further fine-tuned.

In order for it to run on a 1080Ti - @Cognitive mentions the 8-bit variation. It is a quantized 8-bit version - https://huggingface.co/hivemind/gpt-j-6B-8bit - that can run on a single GPU

How many days it would take would depend on your dataset.
Sorry. I wanted to know about the time required for fine-tuning only. How many days would it take to fine-tune the model using 1080Ti approximately?
 

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
Sorry. I wanted to know about the time required for fine-tuning only. How many days would it take to fine-tune the model using 1080Ti approximately?

Again, it would depend on your dataset. It's very hard to determine the length of time required for a model being trained since there are a lot of moving parts.

Even with the same training dataset, and same configuration - two different systems can result in two different yield times.

It depends on your configuration & the level of optimization at play.
 

prvncqr

Jr. VIP
Jr. VIP
Joined
Jan 3, 2022
Messages
315
Reaction score
107
Again, it would depend on your dataset. It's very hard to determine the length of time required for a model being trained since there are a lot of moving parts.

Even with the same training dataset, and same configuration - two different systems can result in two different yield times.

It depends on your configuration & the level of optimization at play.
okay. Would it take months or days?
 

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
okay. Would it take months or days?

Not in a position to answer this either. Since I don't use local GPU to train models.

On a sidenote, if I were in your place with a dataset that'd take months on a 1080 - I'd invest in some infrastructure. Because by the time you're done training, your dataset would most likely have become obsolete.
 

prvncqr

Jr. VIP
Jr. VIP
Joined
Jan 3, 2022
Messages
315
Reaction score
107
Not in a position to answer this either. Since I don't use local GPU to train models.

On a sidenote, if I were in your place with a dataset that'd take months on a 1080 - I'd invest in some infrastructure. Because by the time you're done training, your dataset would most likely have become obsolete.
Ha ha yes. Can't we use the pretrained model to get fine output? Should we change the prompt? It gives weird outputs sometimes.
 

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
Ha ha yes. Can't we use the pretrained model to get fine output? Should we change the prompt? It gives weird outputs sometimes.

No, the pretrained models are not as optimized in responding to prompts as GPT3.

Also, can we move the questions that require multiple back and forth to the AMA - I'd like for readers to be able to see the updates on this journey.
Everyone should follow this journey, @BlogPro knows what he is talking about :)

ea1LOX7.png
 

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
Ha ha yes. Can't we use the pretrained model to get fine output? Should we change the prompt? It gives weird outputs sometimes.

No, the pretrained models are not as optimized in responding to prompts as GPT3.

Also, can we move the questions that require multiple back and forth to the AMA - I'd like for readers to be able to see the updates on this journey.
Everyone should follow this journey, @BlogPro knows what he is talking about :)

ea1LOX7.png
 

prvncqr

Jr. VIP
Jr. VIP
Joined
Jan 3, 2022
Messages
315
Reaction score
107
No, the pretrained models are not as optimized in responding to prompts as GPT3.

Also, can we move the questions that require multiple back and forth to the AMA - I'd like for readers to be able to see the updates on this journey.


ea1LOX7.png
Okay.
 

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
Link to Previous Updates - #0

Update # 1

Current Progress

Site # 1


- The scrapers have begun their job.
- About 2000 questions imported, cleaned and sanitized.

9YzROot.jpeg

- Running a combination script, so the scraping + WordPress publishing and any other AI Augmentation will happen together.

//

Site # 2

- Scraper just deployed

//

Site # 3

- Scraper just deployed

//

Site # 4

- Domain Registered
- Keyword list will need some more sanitization, finding tons of contextually irrelevant ones.

//

Site # 5

- SAO
 

Endonici

Junior Member
Joined
Nov 13, 2021
Messages
101
Reaction score
17
I run several websites - both whitehat and blackhat. I have primarily been doing PPC, CPA, Lead-gen and a little of Adsense. This journey will finally propel me to start using Display.
What CPA Network are you using?
 

BlogPro

Jr Vip
Jr. VIP
Joined
Apr 23, 2012
Messages
1,626
Reaction score
2,459
Link to Previous Updates - #0 | #1

Update # 2

Current Progress

Site # 1


- 6000 PAA Posted + Scheduled
- IndexingAPI Configured
- More on the way.


//

Site # 2

- 3250+ PAA scraped
- 240+ Augmented Content Post Created + Scheduled
- About 2000 questions imported, cleaned and sanitized.
- More on the way.


//

Site # 3

- 7000+ PAA Scraped
- This is the twist, and it is considerably slower when generating content + posts than normal PAA only.
- IndexingAPI Configured
- More on the way.

//

Site # 4

- Keyword List cleaned
- Scraper just Deployed
- Had some problems with proxies being down. I always intend to mix and match proxies from a few different providers. And I always forget. Have pinged the provider and added a new set of proxies.


//

Site # 5

- Content Production Begins
- 190 Posts Created + Scheduled
- More on the way.
 

callmemaximillian

Regular Member
Joined
Feb 9, 2016
Messages
253
Reaction score
91
very interesting journey, doing semi-automated article generation right now.
Think that will learn a lot from your journey
 
Top