0
点赞
收藏
分享

微信扫一扫

Python字符串模糊匹配:thefuzz


在查询数据时,字符串匹配经常用到模糊匹配,这时就要用到模糊匹配算法,如Levenshtein Distance 算法,计算编辑距离,这里Python的thefuzz包实现了模糊匹配功能。

安装

pip install thefuzz

使用

简单匹配

from thefuzz import fuzz

rt = fuzz.ratio("我在山东", "我在山东省")
print(rt)  # 89

非完全匹配

非完全匹配,精度较高。

from thefuzz import fuzz

rt = fuzz.partial_ratio("我在山东", "我在山东省")
print(rt)  # 100

忽略顺序匹配(Token Sort Ratio)

from thefuzz import fuzz

rt = fuzz.token_sort_ratio("我在 山东", "山东 我在")
print(rt)  # 100

去重子集匹配(Token Set Ratio)

from thefuzz import fuzz

rt = fuzz.token_set_ratio("我在 山东", "山东 我在 山东")
print(rt)

extract提取多条数据

from thefuzz import process

choices = ["山东", "山东省", "青岛市", "青岛"]
ls = process.extract("青岛", choices, limit=2)
print(ls)
# [('青岛', 100), ('青岛市', 90)]

如果只提取一条,limit参数可以改为1,或者用extractOne()

from thefuzz import process

choices = ["山东", "山东省", "青岛市", "青岛"]
ls = process.extractOne("青岛", choices)
print(ls)

Pandas Dataframe字段对比

from thefuzz import process


# 模糊匹配

def fuzzy_merge(df_1, df_2, key1, key2, threshold=90, limit=2):
    """
    :param df_1: the left table to join
    :param df_2: the right table to join
    :param key1: key column of the left table
    :param key2: key column of the right table
    :param threshold: how close the matches should be to return a match, based on Levenshtein distance
    :param limit: the amount of matches that will get returned, these are sorted high to low
    :return: dataframe with boths keys and matches
    """
    s = df_2[key2].tolist()

    m = df_1[key1].apply(lambda x: process.extract(x, s, limit=limit))
    df_1['matches'] = m

    m2 = df_1['matches'].apply(
        lambda x: [i[0] for i in x if i[1] >= threshold][0] if len([i[0] for i in x if i[1] >= threshold]) > 0 else '')
    df_1['matches'] = m2

    return df_1


df = fuzzy_merge(df1, df2, '字段1', '字段2', threshold=90)


举报

相关推荐

0 条评论