Laxman Vijay

twitter

facebook

instagram

linkedin

Your privacy is important to us. We promise not to send you spam!

Get inspired from our coverage of the latest trends and breakthroughs in the world of gaming and AI.

Learn the newest gaming tech updates with our weekly newsletter


Bard is Google’s version of conversational AI similar to ChatGPT which has taken the internet by storm over the past few months. This is due to the nature of human like conversational abilities by these AI systems, more and more people finding it using for a variety of tasks.

When it comes to conversational AI, Google is undisputed given the amount of research and development they have done over the years in this field. Bard is the next addition in their lineup. Although given the timing, Bard is simply recognised as a competitor to ChatGPT rather than a flagship product from Google.

Bard is based on the Language Model for Dialogue Application or LaMDA which was released 2 years also by Google. 

“Bard seeks to combine the breadth of the world’s knowledge with the power, intelligence and creativity of our large language models. It draws on information from the web to provide fresh, high-quality responses” according to the [announcement blog](https://blog.google/technology/ai/bard-google-ai-search-updates/) from Google.

This new product adds to Google’s portfolio of AI models including LaMDA, PaLM, Imagen and MusicLM. All these provides more tools to enhance generational AI and impacting the creativity space. 

As of now, there is no demo version of Bard similar to how [ChatGPT has one.](http://chat.openai.com) However, Google says, these new features will be integrated into Google search soon. 

Also, Google plans to release developer APIs from next month starting with Language API powered by LaMDA and followed by other models soon.

Google introduces Bard which is a conversational AI similar to ChatGPT.

Google introduces Bard, Its answer to ChatGPT


Last month Meta [announced they are releasing a new LLM model](https://www.archivorous.com/posts/meta-announces-llama-llm-model) for research purposes. The model named LLaMA(Large Language Model Meta AI) is a state of the art LLM to help researchers run large models in a local environment (possibly in a single machine) to impact various research areas in the LLM domain.

This week Bulgaria based ****[Georgi Gerganov](https://github.com/ggerganov)** has released a [repo](https://github.com/ggerganov/llama.cpp) to execute the model in a M1 Mac. The port called as LLaMA.cpp uses 4 bit quantization to reduce the memory requirements of the model and run in a single Mac device. 

Running the actual LLaMA requires the following steps mentioned in the official repo: https://github.com/facebookresearch/llama. A request must be made to Meta to download the weights for the model. [Here is the official Google form from Meta](https://docs.google.com/forms/d/e/1FAIpQLSfqNECQnMkycAp2jP4Z9TFX0cGR4uf7b_fBxjY_OjhJILlKGA/viewform?usp=send_form) to request to download the weights.

Although the weights have leaked online. It is not legal to download and use this for commercial purpose. Use this magnet to download the weights:

```tsx
magnet:?xt=urn:btih:b8287ebfa04f879b048d4d4404108cf3e8014352&dn=LLaMA&tr=udp%3a%2f%2ftracker.opentrackr.org%3a1337%2fannounce
```

> Note: The magnet above is posted by one of the original requesters to which Meta has sent a download link. Technically, it is still not legal to download files that is intended for use by someone else but it should be fine for experiments and never for commercial use.
> 

> Refer LLaMA model card for info: [llama/MODEL_CARD.md at main · facebookresearch/llama · GitHub](https://github.com/facebookresearch/llama/blob/main/MODEL_CARD.md)
> 

Here are the steps to run the model after you obtained the weights:

- Clone the repo:

```
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
```

- Place the weights in a model folder: (obtained from above methods)

```tsx
ls ./models
65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model
```

- Install dependencies:

```tsx
python3 -m pip install torch numpy sentencepiece
```

- Convert to ggml format: (which is a tensor format developed by the same author)

```tsx
python3 convert-pth-to-ggml.py models/7B/ 1
```

- Quantize:

```tsx
./quantize.sh 7B
```

- Run inference:

```tsx
./main -m ./models/7B/ggml-model-q4_0.bin -t 8 -n 128
```

- Profit!

## The above steps are just too much!

If you feel you don’t want to hustle with the above steps, there is a simple alternative in this repo: [cocktailpeanut/dalai: The simplest way to run LLaMA on your local machine (github.com)](https://github.com/cocktailpeanut/dalai)

Simple run:

```tsx
npx dalai llama
npx dalai serve
```

- Visit [http://localhost:3000](http://localhost:3000/)

Above two commands do the following:

1. First installs the 7B module (default)
2. Then starts a web/API server at port 3000

Interesting to watch what people come up with using LLaMA!

It is now possible to run Meta's LLaMA LLM model locally in a M1 Mac!

Meta

Run Meta’s LLaMA locally in a Macbook

Follow us

Most read articles

Learn the newest gaming tech updates with our weekly newsletter