0
点赞
收藏
分享

微信扫一扫

【vue/uniapp】使用 uni.chooseImage 和 uni.uploadFile 实现图片上传(包含样式,可以解决手机上无法上传的问题)

无愠色 2024-01-04 阅读 12

将excel文件转成txt文件,再将txt文件上传hdfs,最后传入hive中

注意的点
(1)先判断写入的txt文件是否存在,如果不存在就需要创建路径
(2)如果txt文件已经存在,那么先将对应的文件进行删除后再写入txt数据
(3)excel文件中有可能第一行是字段名,需要跳过
1.利用python将excel转成txt文件

from datetime import datetime, timedelta
import os
import pytz
import pandas as pd

def excel_to_txt(name, date):
    # Read Excel file into a DataFrame
    df = pd.read_excel(f'data/excel/{name}.xlsx', header=None, skiprows=1)

    # Define output directory and path
    output_directory = os.path.join('data', 'txt', date)
    os.makedirs(output_directory, exist_ok=True)  # Create directory if it doesn't exist
    output_path = os.path.join(output_directory, f'{name}.txt')

    # Check if the file already exists, if so, remove it
    if os.path.exists(output_path):
        os.remove(output_path)
        print(f'Existing file {output_path} removed.')

    # Write DataFrame to a new text file
    print('开始写入txt文件')
    df.to_csv(output_path, header=None, sep='\t', index=False)
    print('文件写入成功!')
    return output_path



if __name__ == '__main__':
    current_time = datetime.now(pytz.timezone('Asia/Shanghai'))
    one_day_ago = (current_time - timedelta(days=1)).strftime('%Y-%m-%d')
    local_file_path = excel_to_txt('IS_GS_Recruitment_Data_20231211', one_day_ago)
    print(local_file_path)

2.上传到hdfs
3.在hive中创建表

drop table if exists ticket.test_text;
create external table IF NOT EXISTS ticket.test_text
(
    name string,
    age int
) comment ''
      row format delimited fields terminated by '\t'
    lines terminated by '\n'
    NULL DEFINED AS ''
    stored as textfile
    LOCATION '/warehouse/ticket/ods/test_text';

4.将hdfs数据写入hive

load data inpath '/origin_data/test.txt' overwrite into table ticket.test_text;
举报

相关推荐

uni-app图片上传和预览

0 条评论