Awesome

GeoGalactica Analysis

This repo is for people who want to draw training curves, weight value hist and heat maps.

Training Curve

From tensorborad event log draw traing cure (including loss, learning rate, grad norm curve)

export tensorboard_path=""
python draw_training_curve --tensorboard_path ${tensorboard_path} 

Weight Hist

1. save model’s weight

   prepare qkv/ out/ fc weight for

export layer=0
export lla_model_path=""
export gla_model_path=""
python save_weight_value.py --lla_model_path ${lla_model_path} --gla_model_path ${gla_model_path} --layer ${layer}

2. draw weigth hist

export weight_type="" # [qkv, out, fc]
python draw_hist.py --weight_type ${weight_type} --layer 0

Heat Map

1. calculate the each categlory prompt similarty.

export prompt_path = ""
python cal_prompt_simliarty.py --prompt ${prompt_path}

2. draw heat map

export similarity_matrix="" # [qkv, out, fc]
python draw_heat_map.py --similarity_matrix ${similarity_matrix}