Fork me on GitHub

快速删除大量文件

生成大量文件 mkdir lostfiles cd lostfiles cat create_files.sh for i in $(seq 1 1500000) do echo test >>$i.txt done time sh create_files.sh real 3m44.841s user 0m13.208s sys 3m14.523s 快速删除方式对比 time rsync --delete-before -d /tmp/null/ lostfiles/ real 1m14.937s user 0m1.769s sys 0m54.957s time rm lostfiles/ -rf real 1m4.221s user 0m0.776s sys 0m40.334s time find ./lostfiles/ -type f -delete real 1m0.695s user 0m0.851s sys 0m41.294s time perl -e 'for(<*>){((stat)[9]<(unlink))}' real 1m9.959s user 0m2.955s sys 0m50.202s 应用 快速清除minio bucket 数据 清除bucket数据 time find ./bucket001/ -type f -delete 清除bucket元数据 time find .minio.sys/buckets/bucket001 -type f -delete……

阅读全文

perf linux 性能分析

收集数据 sudo perf record -F 99 -p 13204 -g -- sleep 30 -F 99表示每秒99次, -p 13204是进程号 -g表示记录调用栈 sleep 30则是持续30秒 数据解析 perf script -i perf.data &> perf.unfold 数据展现 git clone https://github.com/brendangregg/FlameGraph.git 将perf.unfold中的符号进行折叠 ./stackcollapse-perf.pl perf.unfold &> perf.folded 生成svg图 ./flamegraph.pl perf.folded > perf.svg 用浏览器打开查看 结果解读 https://www.ruanyifeng.com/blog/2017/09/flame-graph.html……

阅读全文

高压缩比工具 XZ

压缩比 xz >biz2 > gzip 安装 默认系统自带 yum install epel-release yum install xz 解压缩 xz -d --threads=`nproc` -k -v hits_v1.tsv.xz 压缩 xz -z --threads=`nproc` -k -v hits_v1.tsv 参数说明 # xz --help Usage: xz [OPTION]... [FILE]... Compress or decompress FILEs in the .xz format. -z, --compress force compression -d, --decompress force decompression -t, --test test compressed file integrity -l, --list list information about .xz files -k, --keep keep (don't delete) input files -f, --force force overwrite of output file and (de)compress links -c, --stdout write to standard output and don't delete input files -0 ... -9 compression preset; default is 6; take compressor *and* decompressor memory usage into account before using 7-9! -e, --extreme try to improve compression ratio by using more CPU time; does not affect……

阅读全文

Linux 找出隐藏进程

原理 在top ps 等命令被改写时,利用Linux一切皆文件。找出被隐藏的进程。 以下为python脚本 #!/usr/bin/env python # -*- coding: utf-8 -*- import os def get_max_pid(): out = os.popen('cat /proc/sys/kernel/pid_max') content = out.readline().strip('\n') if content.isdigit(): return int(content) def get_ps_proc_list(): pid_list = [] out = os.popen('ps -e --no-header') lines = out.readlines() for line in lines: parts = line.split(' ') for part in parts: if part == '': parts.remove(part) pid = int(parts[0]) pid_list.append(pid) return pid_list def get_ps_lwp_list(): lwp_list = [] out = os.popen('ps --no-header -eL o lwp') lines = out.readlines() for line in lines: tid = int(line) lwp_list.append(tid) return lwp_list def print_badpid_info(pid): out = os.popen('ls -l /proc/%d/exe' % pid)……

阅读全文

Linux 系统登陆记录

背景 登陆系统时,尤其是具有外网ip的主机时经常会看到类似如下信息。 There were 12039 failed login attempts since the last successful login. 说明你的系统被尝试登陆破解。 大部分的破解基本都是自动机器扫描,配合自己的数据字典暴力破解。 系统登陆成功记录 查看命令 last 原理 读取解析 /var/log/wtmp 例如: 查看最近十次登陆记录 last -10 查看某个时间段的登陆记录 last -s 2021-10-10 -t……

阅读全文

Ubuntu20.04 装机后

关闭cloud init systemctl stop cloud-init-local cloud-init cloud-config cloud-final systemctl disable cloud-init-local cloud-init cloud-config cloud-final……

阅读全文