AI embeddings, LORA, RAG, rope based frequency

 Embeddings

https://huggingface.co/blog/getting-started-with-embeddings

https://www.sbert.net

https://medium.com/enterprise-rag/a-first-intro-to-complex-rag-retrieval-augmented-generation-a8624d70090f

========================

Google: RAG

https://medium.com/enterprise-rag/a-first-intro-to-complex-rag-retrieval-augmented-generation-a8624d70090f

https://www.griddynamics.com/blog/retrieval-augmented-generation-llm

https://m.youtube.com/watch?v=TRjq7t2Ms5I

https://llamahub.ai

======================

Google: Lora

https://m.youtube.com/watch?v=dA-NhCtrrVE&t=6s&pp=2AEGkAIB

https://m.youtube.com/watch?v=iYr1xZn26R8

https://arxiv.org/abs/2106.09685

========================

Google: llama model rtx 3060 6gb

https://www.hardware-corner.net/guides/computer-to-run-llama-ai-model/

https://huggingface.co/TheBloke/OpenHermes-2.5-Mistral-7B-GGUF

https://medium.com/@phillipgimmi/what-is-gguf-and-ggml-e364834d241c

=======================

Google: Andrej Karpathy’s course on building and training a tokenizer!

https://medium.com/the-research-nest/explained-tokens-and-embeddings-in-llms-69a16ba5db33

https://datasciencedojo.com/blog/embeddings-and-llm/

Google: llm embeddings

https://dencode.com/en/string/unicode-normalization

https://www.w3schools.com/python/ref_dictionary_get.asp

https://tiktokenizer.vercel.app/?encoder=gpt2

https://ilampard.github.io/blog/2024/tokenizer/

https://arxiv.org/abs/1804.10959

https://aclanthology.org/P16-1162/

https://github.com/google/sentencepiece

https://github.com/karpathy/minbpe/blob/master/exercise.md

https://platform.openai.com/tokenizer

https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb

https://github.com/python/typeshed

https://github.com/karpathy/minbpe/blob/master/minbpe/base.py

https://www.lesswrong.com/posts/aPeJE8bSo6rAFoLqg/solidgoldmagikarp-plus-prompt-generation

https://en.m.wikipedia.org/wiki/Byte_pair_encoding#

https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf

https://francescopochetti.com/byte-pair-encoding-building-the-gpt-tokenizer-with-karpathy/

https://karpathy.ai/zero-to-hero.html

============================

Google: rope-freq-basegoogle: rope based frequency 

Google rope based frequency 

https://github.com/ggerganov/llama.cpp/discussions/1965

https://arxiv.org/pdf/2207.04901

https://arxiv.org/abs/2306.15595

https://kaiokendev.github.io/context

https://kaiokendev.github.io/til#extending-context-to-8k

https://kaiokendev.github.io/til#extending-context-to-8k

Attachments area

Preview YouTube video Building Production-Ready RAG Applications: Jerry Liu

Building Production-Ready RAG Applications: Jerry Liu

Preview YouTube video Low-rank Adaption of Large Language Models: Explaining the Key Concepts Behind LoRA

Low-rank Adaption of Large Language Models: Explaining the Key Concepts Behind LoRA

Preview YouTube video Low-rank Adaption of Large Language Models Part 2: Simple Fine-tuning with LoRA