区块链技术及其金融应用

BTC区块链原理

资源¶

  • 资讯

    • 网信办区块链服务备案系统
    • 国家信用信息公示系统
    • 天眼查
    • 巨潮资讯网
  • 软件

    • Hyperledger
    • R3
    • Corda
  • BaaS

Content¶

  • ## 金融科技与金融科技项目
    • ### 金融科技
    • ### 金融科技项目
    • ### 区块链技术及其金融应用
  • ## 比特币与区块链
    • ### 比特币的起源与发展
    • ### 比特币区块链
    • ### 实验:BTC区块分析
    • ### 区块链技术的应用

金融科技
¶

金融科技的定义¶

Financial Stability Board (FSB) "Monitoring of FinTech"¶

The FSB understands FinTech as technologically enabled innovation in financial services that could result in new business models, applications, processes or products with an associated material effect on financial markets and institutions and the provision of financial services.

中国人民银行《金融科技(FinTech)发展规划(2019-2021年)》¶

金融科技是技术驱动的金融创新(该定义由金融稳定理事会(FSB)于2016年提出,目前已成为全球共识),旨在运用现代科技成果改造或创新金融产品、经营模式、业务流程等,推动金融发展提质增效。

Schueffel(2017)¶

After reviewing more than 200 scientific papers citing the term "fintech," a study on the definition of fintech concluded that "fintech is a new financial industry that applies technology to improve financial activities."

Investopedia¶

Financial technology (Fintech) is used to describe new tech that seeks to improve and automate the delivery and use of financial services.

CFA ® Program Curriculum¶

In its broadest sense, the term “fintech” generally refers to technology-driven innovation occurring in the financial services industry. For the purposes of this reading, fintech refers to technological innovation in the design and delivery of financial services and products.

CCB¶

Fintech=A(rtificial Intelligence)+B(igData)+C(loud Computing)+D(LT)+M(obile Internet)+I(oT)+X

Fintech应用领域:Investopedia¶

  • Cryptocurrency and digital cash
  • Blockchain technology, including Ethereum, a distributed ledger technology (DLT) that maintain records on a network of computers, but has no central ledger.
  • Smart contracts, which utilize computer programs (often utilizing the blockchain) to automatically execute contracts between buyers and sellers.
  • Open banking, a concept that leans on the blockchain and posits that third-parties should have access to bank data to build applications that create a connected network of financial institutions and third-party providers. An example is the all-in-one money management tool Mint.
  • Insurtech, which seeks to use technology to simplify and streamline the insurance industry.
  • Regtech, which seeks to help financial service firms meet industry compliance rules, especially those covering Anti-Money Laundering and Know Your Customer protocols which fight fraud.
  • Robo-advisors, such as Betterment, utilize algorithms to automate investment advice to lower its cost and increase accessibility.
  • Unbanked/underbanked, services that seek to serve disadvantaged or low-income individuals who are ignored or underserved by traditional banks or mainstream financial services companies.
  • Cybersecurity, given the proliferation of cybercrime and the decentralized storage of data, cybersecurity and fintech are intertwined.

Fintech应用领域:CFA ® Program Curriculum¶

  • Analysis of large datasets.
  • Analytical tools. For extremely large datasets, techniques involving artificial intelligence (AI)。
  • Automated trading.
  • Automated advice (Robo-advisers)
  • Financial record keeping (DLT)

大数据¶

  • Volume: The amount of data collected in files, records, and tables is very large, representing many millions, or even billions, of data points.
  • Velocity: The speed with which the data are communicated is extremely great. Real- time or near- real- time data have become the norm in many areas.
  • Variety: The data are collected from many different sources and in a variety of formats, including structured data (e.g., SQL tables or CSV files), semi-structured data (e.g., HTML code), and unstructured data (e.g., video messages).
  • Value

数据科学¶

  • Capture—Data capture refers to how the data are collected and transformed into a format that can be used by the analytical process. Low-latency systems—systems that operate on networks that communicate high volumes of data with minimal delay (latency)—are essential for automated trading applications that make decisions based on real- time prices and market events. In contrast, high-latency systems do not require access to real- time data and calculations.
  • Curation—Data curation refers to the process of ensuring data quality and accuracy through a data cleaning exercise. This process consists of reviewing all data to detect and uncover data errors—bad or inaccurate data—and making adjustments for missing data when appropriate.
  • Storage—Data storage refers to how the data will be recorded, archived, and accessed and the underlying database design. An important consideration for data storage is whether the data are structured or unstructured and whether analytical needs require low-latency solutions.
  • Search—Search refers to how to query data. Big Data has created the need for advanced applications capable of examining and reviewing large quantities of data to locate requested data content.
  • Transfer—Transfer refers to how the data will move from the underlying data source or storage location to the underlying analytical tool. This could be through a direct data feed, such as a stock exchange’s price feed.
  • Visualization

一些应用¶

  • Text Analytics
    • Text analytics involves the use of computer programs to analyze and derive meaning typically from large, unstructured text- or voice-based datasets, such as company filings, written reports, quarterly earnings calls, social media, email, internet postings, and surveys.
    • Text analytics includes using computer programs to perform automated information retrieval from different, unrelated sources in order to aid the decision-making process.
    • More analytical usage includes lexical analysis, or the analysis of word frequency in a document and pattern recognition based on key words and phrases.
    • Text analytics may be used in predictive analysis to help identify indicators of future performance, such as consumer sentiment.
  • Natural Language Processing
    • Natural language processing (NLP) is a field of research at the intersection of computer science, artificial intelligence, and linguistics that focuses on developing computer programs to analyze and interpret human language.
    • Automated tasks using NLP include translation, speech recognition, text mining, sentiment analysis, and topic analysis.
  • Other Application
    • Robo-Advisory Services
    • Risk Analysis
    • Algorithmic Trading

Science, Technology, and Engineering¶

Science Technology Engineering
Science (from the Latin word scientia, meaning "knowledge") is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions about the universe. Modern science is typically divided into the natural sciences, the social sciences and the formal sciences. Disciplines that use existing scientific knowledge for practical purposes, such as engineering and medicine, are described as applied sciences. Technology ("science of craft", from Greek τέχνη, techne, "art, skill, cunning of hand"; and -λογία, -logia) is the sum of techniques, skills, methods, and processes used in the production of goods or services or in the accomplishment of objectives, such as scientific investigation. The creative application of scientific principles to design or develop structures, machines, apparatus, or manufacturing processes, or works utilizing them singly or in combination; or to construct or operate the same with full cognizance of their design; or to forecast their behavior under specific operating conditions; all as respects an intended function, economics of operation and safety to life and property.

金融科技项目¶

培养目标¶

  • 理解金融业务逻辑
  • (略)懂信息技术
  • 会应用(新的技术、方法、模型)

毕业论文¶

  • 案例分析
  • 产品设计与金融实践问题解决方案
  • 调研报告(或基于实际问题分析的政策建议报告)等

案例分析¶

  • #### 第一部分:绪论。
    • 内容一般包括研究背景和意义(包括案例的典型性和代表性说明)、相关研究综述、研究方法、创新点、论文的结构。
  • #### 第二部分:案例介绍。

    • 介绍概况、历史沿革,与选题相关内容的现状。案例介绍是案例中一段既定的历史,即在案例写作之前已经发生的事实,包括问题本身及已经采取的解决办法等。
  • #### 第三部分:案例存在的问题和原因分析。

    • 案例资料是案例分析的基石。案例资料的获取可以通过文件、档案记录、访谈、直接观察、参与性观察和实物证据等。资料收集过程中,要尽量使用多种数据来源,并注意及时记录和整理相关资料,原则上要求原创性资料(一手资料)比例不少于60%。

    • 在已获得充实的资料的基础上选用适合的方法进行分析,如图表分析、因果分析和对比分析等,运用相关金融理论和知识找出案例的启示及存在的问题。

  • #### 第四部分:解决问题和改进建议。

产品设计¶

  • #### 第一部分:产品方案基本概念和设计理念

    • 介绍产品的核心特征、功能。金融产品的设计理念,要求在全面的风险和收益分析上体现产品的创新之处。
  • #### 第二部分:设计方案

    • 方案设计主要包括产品功能与特点介绍、产品的风险与收益、产品创新点分析、产品市场分析。
  • #### 第三部分:产品优势

    • 主要反映在:成本收益比较、与类似产品的优劣比较、符合市场趋势等方面。
  • #### 第四部分:产品推广策略

金融实践问题解决方案¶

  • #### 第一部分:问题描述

    • 对金融实践问题进行描述、介绍问题产生的背景。
  • #### 第二部分:分析问题

    • 金融实践问题的现状、产生的原因、问题的发展趋势分析、解决问题的必要性与现实价值、国内外类似问题及其解决方案情况。
  • #### 第三部分:问题解决方案设计

    • 方案设计的理念、方案设计的条件、方案设计的具体内容、方案的特点与创新点、方案实施的核心点、设计方案存在的风险分析。
  • #### 第四部分:方案的合理性论证(检验)以及实施途径

    • 设计方案的可行性分析、成本收益分析、方案实施中可能出现的问题及其应对措施。

调研报告¶

  • #### 报告正文的内容要求:

    • 正文部分为作者所要论述的主要事实和观点,包括介绍调研针对的问题、目的和意义、相关背景、时间、地点、人员、调查手段,以及对实践活动中得到的结论的详细叙述。写作上要求能够体现金融理论的思想路线,有新观点、新思路;要理论联系实际,必须以实证的材料、数据、统计等为支撑,对实际工作有指导作用和借鉴作用,能提出建设性意见和建议;内容观点鲜明,重点突出,结构合理,逻辑清晰,文字通畅、精炼。
  • #### 报告正文的逻辑结构形式:

    • “现状–问题–解决方案–建议”式结构——多用于反映、解决问题的报告;

    • “成果–具体做法–经验”式结构——多用于总结经验的报告;

    • “事件过程–事件性质结论–处理意见”式结构——多用于揭示是非的报告。

区块链技术及其金融应用¶

教学计划¶

课程教学计划¶

时间 教学内容
wk10-wk12 区块链技术基础与比特币区块链
wk13-wk15 以太坊区块链及其金融应用
wk16-wk17 数字加密货币相关问题

前三周教学计划¶

  • 区块链基本原理
  • (比特币、以太坊、Hyperledger Fabric)区块链的应用
  • 关于区块链与数字货币的研究
  • 课程项目分享与点评

考核方式:课程项目¶

  • 项目内容:区块链案例分析、区块链产品设计、区块链研究设计等
  • 提交内容:报告+PPT+视频(10分钟)
  • DDL:见课程主页
  • 提交方法:见课程主页

比特币与区块链
¶

比特币的起源与发展
¶

BTC的发展历史)¶

比特币的诞生与技术基础¶

  • 诞生
    • 中本聪(Satoshi Nakamoto)发布白皮书(2008.10.31):Bitcoin: A Peer-to-Peer Electronic Cash System
    • 2019年1月开源比特币代码,比特币上线运行
    • 背景:次贷危机
  • 思想来源
    • Friedrich Hayek: The Denationalization of Money (1976)
  • 技术基础
    • P2P技术
    • 加密技术
    • 共识机制
    • 博弈论/激励机制

思考:如何理解货币形态的演变?¶

不同形态的比特币¶

  • 通常只是一串代码
    • 体现为帐簿里某一地址下未花费的交易输出
  • 实体比特币
    • 由Casascius公司发行
    • 内置含有密钥的芯片

  • 比特币借记卡
    • 由Xapo有限公司发行
    • 被认为是全球第一张比特币借记卡
    • 与普通借记卡类似的用户体验

比特币的价格与价值¶

两块披萨¶

  • On May 22, 2010, Laszlo Hanyecz agreed to pay 10,000 Bitcoins for two delivered Papa John's pizzas.
  • Two pizzas worth $25 at that time.

披萨的价值¶

2010 ... 2013 2014 2015 2016 2017 2018 2019 2020 2021 今天
BTC/USD 0.0025 ... 123.89 524.58 240.35 439.32 2,173.40 8,041.78 7,680.07 6,452.69 37,536.63 39,073.16
Pizza in $ 25 ... 1.2mi 5.2mi 2.4mi 4.4mi 21.7mi 80.4mi 76.8mi 64.5mi 375.37mi 390.73mi
USD/CNY 6.8298 ... 6.1312 6.2352 6.1974 6.549 6.8863 6.3661 6.9061 7.09 6.43 6.49
Pizza in ¥ 170.75 ... 760万 3271万 1490万 2877万 1亿4967万 5亿1195万 5亿3039万 4亿5750万 24亿1363万 25亿3585万
  • BTC/USD reached 67,673.74 on November 8th, 2021. The two pizzas worth 676.74 million dollars, or equivalently 43.28亿 in CNY at that time.
  • He is the first man, who utilizes GPU in BTC mining.

比特币披萨日¶

研究:数字加密货币的价值¶

Buffett on cryptocurrencies¶

"In terms of cryptocurrencies generally, I can say almost with certainty that they will come to a bad ending," Buffett said in an interview with CNBC in January 2018. "Now, when it happens or how or anything else, I don't know."

He added: "If I could buy a five-year put on every one of the cryptocurrencies, I'd be glad to do it, but I would never short a dime's worth."

你怎么看待数字加密币的价值?¶

Do Fundamentals Drive Cryptocurrency Prices?¶

Two Types of Cryptocurrencies¶

  • Mineable cryptocurrencies are rewards to solving a cryptographic algorithm via a process known as mining

  • The distribution and creation of non-mineable cryptocurrencies (NMCs) is decided ex-ante and is generally based on the protocol of their founders.

  • This paper focus on mineable cryptocurrencies

Two Important Factors¶

  • Computing Power

    • 它促进了交易的快速和安全记录
    • 以hash rate来衡量
    • 与矿工在区块链中创建块所消耗的资源直接相关
  • Network

    • 网络测量每天在区块链上交易加密货币的唯一地址的数量
    • 大量的区块链用户也暗示了各自加密货币的流动性增强
    • 大型网络吸引了开发人员为该加密货币的区块链构建应用程序,从而提高了该货币的可用性。

文献:BTC是不是货币?¶

  • Yermack (2015比特币不是一种真正的货币,因为它不是一种广泛使用的交换媒介或记账单位。他的结论是,由于比特币相对于其他货币的价格波动较大,它更像是一种投机性资产。
  • Selgin (2015) and Baur et al. (2018)认为比特币结合了法定货币和商品货币的特点。
  • Eichengreen (2019)回顾了现代货币的经济史,认为加密货币没有提供货币的基本功能。
  • Pagano and Sedunov (2019)认为比特币是一种真正的货币形式。

文献:影响BTC价格的因素¶

  • Auer and Claessens (2018)认为由于市场分割:每个管辖区的监管新闻仅对该管辖区的加密货币价格有影响。
  • Makarov and Schoar (2018)研究了不同交易所比特币,以太坊,瑞波币的价格,他们发现有时候不同交易所的价格不同,表明有巨大的套利利润,导致持续的价格走高和跳跃。
  • Cheah and Fry (2015) 发现2010年7月到2014年7月,比特币经历了一个泡沫时期。
  • Cheah and Fry (2015)认为比特币基本价值为0,它的价格受投资者情绪驱动

文献:加密币的定价问题¶

  • Baur, Hong, and Lee (2018) 发现比特币与其他资产不相关,如股票,债券,商品。
  • Liu and Tsyvinski (2018) 展示了比特币,Ripple和以太坊与股票,债券,货币和贵金属市场无关。 他们还发现,加密货币不受现有金融因素影响,而是受两种特定于加密货币的因素影响。这两种因素是动量(被滞后的加密货币回报反映)和投资者关注(被谷歌搜索和推特活动反映)影响.
  • Liu, Tsyvinski,and Wu (2019) 对一组更大数量的加密货币得出了类似的结论,并且还强调了基于加密货币的规模因素的重要性。
  • Bianchi (2018)也发现了在比特币回报中有强烈的动量效应
  • Pagnotta和Buraschi(2018)从理论的角度研究了一般均衡模型下的可开采加密货币。它们表明,加密货币的均衡基本价值受到消费者偏(反风险和审查)和货币的有用性(由信任和采用水平反映)
  • Biais等人(2018)构建了一个重叠代际模型,其中加密货币的价值依赖于与加密货币用户数量相关的交易收益。他们利用模型来研究比特币。他们发现,他们发现比特币的回报取决于比特币的交易费用和与将比特币与其他货币和消费品进行兑换的难易程度相关的事件.

假设¶

  • 在单个加密货币水平上,加密货币价格应该平均来说与计算能力和网络大小有正的相关关系
  • 在加密货币市场层面上,我们研究了区块链基本面的总体衡量指标是否捕获了系统风险的来源

变量¶

price Average of daily log prices over the 7-day period ending on Friday. The daily price is the fixed closing price at midnight UTC of the current day denominated in U.S. dollars. The daily prices are from Coinmetrics.io’s fixing/reference rate service.
CP Average of daily log hashrate over the 7-day period ending on Friday. The daily hasharate is the mean difficulty of finding a new block multiplied by the number of blocks mined on that day. For example, if 100 blocks of Bitcoin were mined on a particular day and each block required an average of 5 TH to mine, the computing power value would be 500 × 1012 hashes.
NET Average of daily log unique active addresses over the 7-day period ending on Friday. The daily addresses are the sum of unique addresses that were active in the network (either as a recipient or originator of a ledger change) that day. Unique active addresses are the number of addresses from (or to) which transactions are conducted on the cryptocurrency’s respective blockchain. Network data for Monero are not available.

Computing Power¶

Network¶

Common Risk Factors in Cryptocurrency (JF2022)¶

Cross-Sectional Factors

  • Size Factor Returns

  • Momentum Factor Returns

  • Volume Factor Returns

  • Volatility Factor Returns

Cryptocurrency Factors

  • Cryptocurrency One-Factor Model

  • Cryptocurrency Market and Size Factor Model

  • Cryptocurrency Market and Momentum Factor Model

  • Cryptocurrency Three-Factor Model

Do investor sentiments drive cryptocurrency prices? (Economics Letters 2021)

Is Bitcoin Really Untetherd?

  • Two main alternative hypotheses for Tether
    • Tether is “pulled” (demand-driven): Tether is driven by legitimate demand from investors who use Tether as a medium of exchange to enter their fiat capital into the cryptospace because it is digital currency with the stability of the dollar “peg.” In this case, the price impact of Tether reflects natural market demand.
    • Tether is “pushed” (supply-driven) : Bitfinex prints Tether regardless of the demand from cash investors, and additional supply of Tether can create inflation in the price of Bitcoin that is not due to a genuine capital flow.
  • Motivations for "push"
    • The Tether creators, like most early cryptocurrency adopters and exchanges, have large holdings of Bitcoin, they generally profit from the inflation of the cryptocurrency prices.
    • Coordinated supply of Tether creates an opportunity to manipulate cryptocurrencies—when prices are falling, the Tether creators can convert their large Tether supply into Bitcoin in a way that pushes Bitcoin up and then sell some Bitcoin back into dollars in a venue with less price impact to replenish Tether reserves.
    • If cryptocurrency prices crash, the founders essentially have a put option to default on redeeming Tether, or to potentially experience a “hack” or insufficient reserves where by Tether-related dollars disappear.

Figure 1 Aggregate flow of Tether between major addresses.

  • Main Hypotheses:
    • H1A:Tether's price relative to the USD may increase as a consequence of strong investor demand. Tether flows should be strongly related to this demand as proxied by changes in the Tether-USD exchange rate.
    • H1B:The printing of Tether may also be driven by its usefulness as a facilitator of cross-exchange arbitrage to eliminate pricing discrepancies across cryptocurrency exchanges. For example, Tether outflows from Bitfinex to another exchange should correspond to periods when Bitcoin sells at a premium on Bitfinex relative to that exchange.
  • To examine the “push” hypothesis, the authors test the following predictions:
    • H2A:If Tether issuers are trying to provide stability to the market during downturns, outflows of Tether and purchases of Bitcoin by Bitfinex may follow periods of negative Bitcoin returns.
    • H2B:If Tether supply is large enough to have a material price impact on Bitcoin, Bitcoin prices should go up after Tether flows into the market, especially after periods with large authorization of Tether.
    • H2C:Bitcoin returns may show a return reversal after negative returns, especially during times when Tether flows into the market.
    • H2D:Since round-number thresholds can be price anchors to set a price floor and are often used as buying signals by investors, flow of Tether might increase if Bitcoin falls below these salient round-number thresholds. This effect should be more pronounced in periods with large Tether authorization.
    • H2E:If Tether is not fully backed by dollars at the outset, but the issuers want to signal otherwise to investors by releasing EOM (or other interval) accounting statements, then Tether creators may liquidate Bitcoins into USD to demonstrate sufficient reserves. This could create negative returns in Bitcoin at the EOM, particularly in periods with large Tether issuances.

Figure 5 Prices of Bitcoin and other cryptocurrencies around high-flow events.

Panel A shows Bitcoin prices three hours before and after the top 1% of high-flow hours to Poloniex and Bittrex. Prices are scaled to one at time −3 before the event and at time 0 at the end of the event window. Scaled prices are averaged across events. High-flow events are defined as the top 1% of hours with high net average flows of Tether from Bitfinex to Poloniex and Bittrex and Bitcoin back from Poloniex and Bittrex to Bitfinex in the prior hour, which means that high flows occur between time −1 and time 0. Panel B depicts similar results for other major cryptocurrencies.

比特币区块链 (blockchain v1.0)

BTC区块链的密码学基础¶

Hash¶

  • Hash function:
    • takes any string as input
    • fixed-size output (we’ll use 256 bits)
    • efficiently computable
  • Security properties:
    • collision-free
    • hiding
    • puzzle-friendly

Hash pointer¶

  • hash pointer is:

    • pointer to where some info is stored, and
    • (cryptographic) hash of the info
  • if we have a hash pointer, we can

    • ask to get the info back, and
    • verify that it hasn’t changed

Merkle Tree¶

数字签名¶

  • 生成公钥、私钥

    • (s𝑘,𝑝𝑘):=𝑔𝑒𝑛𝑒𝑟𝑎𝑡𝑒𝐾𝑒𝑦𝑠(𝑘𝑒𝑦𝑠𝑖𝑧𝑒)
  • 签名

    • 𝑠𝑖𝑔≔𝑠𝑖𝑔𝑛(𝑠𝑘, 𝑚𝑒𝑠𝑠𝑎𝑔𝑒)
  • 验证

    • 𝑖𝑠𝑉𝑎𝑙𝑖𝑑≔𝑣𝑒𝑟𝑖𝑓𝑦(𝑝𝑘,𝑚𝑒𝑠𝑠𝑎𝑔𝑒,𝑠𝑖𝑔)
  • 通过验证

    • 𝑣𝑒𝑟𝑖𝑓𝑦(𝑝𝑘,𝑚𝑒𝑠𝑠𝑎𝑔𝑒,𝑠𝑖𝑔𝑛(𝑠𝑘, 𝑚𝑒𝑠𝑠𝑎𝑔𝑒))==𝑇𝑟𝑢𝑒!

Bitcoin consensus algorithm (simplified).¶

This algorithm is simplified in that it assumes the ability to select a random node in a manner that is not vulnerable to Sybil attacks.

    1. New transactions are broadcast to all nodes.
    1. Each node collects new transactions into a block.
    1. In each round, a random node gets to broadcast its block.
    1. Other nodes accept the block only if all transactions in it are valid (unspent, valid signatures).
    1. Nodes express their acceptance of the block by including its hash in the next block they create.

比特币网络是如何实现的?¶

  • 是否可以通过产生大量节点进行攻击?
  • 是否可以窃取比特币(STEALING BITCOINS)?
  • 如何应对拒绝服务攻击(DENIAL-OF-SERVICE ATTACK)?
  • 如何应对双花攻击(DOUBLE-SPEND ATTACK)?

关于bitcoin的研究¶

Bitcoin的黑暗面¶

[11] Foley, Sean, Jonathan R. Karlsen, and Tālis J. Putniņš. "Sex, drugs, and bitcoin: How much illegal activity is financed through cryptocurrencies?." The Review of Financial Studies 32.5 (2019): 1798-1853.

  • Method
    • Identifying a Sample of Illegal Users
      • First approach: Bitcoin seizures by law enforcement agencies
      • Second approach: Illegal darknet marketplaces and their users
      • Third approach: Users identified in darknet forums
    • Quantifying and Characterizing All Illegal Activity
      • Method 1: Network cluster analysis (smart local moving algorithm, SLM)
      • Method 2: Detection-controlled estimation (DCE)
  • Results: How much illegal activity involves bitcoin?

This table reports the size and activity of legal and illegal user groups. The measures of group size and activity are: the number of users (Users), the number of transactions (Transaction count), the average dollar value of bitcoin holdings (Holding value), the number of bitcoin addresses (Number of addresses), and the dollar volume of transactions (Volume). Panel A reports the values of these measures for the two user groups, and panel B expresses the measures for each group as a percentage of the total. Different rows report different approaches to classifying the legal and illegal user groups. SLM provides estimates from the network cluster analysis approach to classification (a variant of the “smart local moving” algorithm). DCE provides estimates from the detection controlled estimation (DCE) approach to classification, which exploits the characteristics of legal and illegal users. Midpoint is the average of the estimates from the SLM and DCE models. Upper bound and Lower bound provide a 99% confidence interval around the Midpoint, using a form of bootstrapped standard errors.

  • Results: How does illegal activity vary through time?

  • Results: How does illegal activity vary through time?

The Scaling Problem of Bitcoin (BTC Blockchain)¶

[3] Hinzen, Franz J., Kose John, and Fahad Saleh. "Bitcoin’s limited adoption problem." Journal of Financial Economics 144.2 (2022): 347-369.

The Scaling Problem of Bitcoin (BTC Blockchain)¶

[27] Malik, Nikhil, et al. "Why Bitcoin will Fail to Scale?." Management Science (2022).

  • the scaling problem
    • Bitcoin throughput is limited by its block capacity
    • approximately 2200 transactions were confirmed every 10 minutes
    • BTC community is considering increase the block capacity
    • The paper proves that it does not work
  • Method
    • a game theoretical model to analyze the strategic interactions of miners and users
  • Results
    • a relaxation of throughput congestion can facilitate large miners to tacitly collude
      • artificially reducing the actualized throughput via the strategic partial filling of blocks to receive higher transaction fees
    • technological intervention (such as banning large miners)
      • can eliminate collusion
      • makes the system less secure

Decentralized Clearing via Blockchain¶

[29] Csóka, Péter, and P. Jean-Jacques Herings. "Decentralized clearing in financial networks." Management Science 64.10 (2018): 4681-4699.

Decentralized Clearing via Blockchain¶

[18] Chiu, Jonathan, and Thorsten V. Koeppl. "Blockchain-based settlement for asset trading." The Review of Financial Studies 32.5 (2019): 1716-1753.

larger trading volume, higher time criticality, and lower default exposure make settlement on a blockchain more attractive.

Decentralized Clearing via Blockchain¶

[22] Cheng, Stephanie F., et al. "Riding the blockchain mania: Public firms’ speculative 8-K disclosures." Management Science 65.12 (2019): 5901-5913.

实验:BTC区块分析¶

1. 用Bitcoin Core同步区块数据+区块解析¶

BTC区块结构(source: arcblock.io)

解析工具¶

  • https://github.com/alecalve/python-bitcoin-blockchain-parser
  • https://github.com/alecalve/python-bitcoin-blockchain-parser/tree/master/blockchain_parser
  • https://github.com/tenthirtyone/blocktools
  • https://github.com/ragestack/blockchain-parser
  • https://github.com/bitcoin-abe/bitcoin-abe
  • https://github.com/huangsuoyuan/btc_parser

2. 网络爬虫¶

爬虫基本原理: HTTP协议¶

网络爬虫基本原理:HTTP请求方法(本页参考W3Cschool相关内容)¶

  • 根据HTTP标准,HTTP请求可以使用多种请求方法。
  • HTTP1.0定义了三种请求方法: GET, POST 和 HEAD方法。
  • HTTP1.1新增了五种请求方法:OPTIONS, PUT, DELETE, TR
序号 方法 描述
1 GET 请求指定的页面信息,并返回实体主体。
2 HEAD 类似于get请求,只不过返回的响应中没有具体的内容,用于获取报头
3 POST 向指定资源提交数据进行处理请求(例如提交表单或者上传文件)。数据被包含在请求体中。POST请求可能会导致新的资源的建立和/或已有资源的修改。
4 PUT 从客户端向服务器传送的数据取代指定的文档的内容。
5 DELETE 请求服务器删除指定的页面。
6 CONNECT HTTP/1.1协议中预留给能够将连接改为管道方式的代理服务器。
7 OPTIONS 允许客户端查看服务器的性能。
8 TRACE 回显服务器收到的请求,主要用于测试或诊断。

网络爬虫基本原理:HTTP状态码分类(本页参考W3Cschool相关内容)¶

HTTP状态码由三个十进制数字组成,第一个十进制数字定义了状态码的类型,后两个数字没有分类的作用。HTTP状态码共分为5种类型:

分类 分类描述
1** 信息,服务器收到请求,需要请求者继续执行操作
2** 成功,操作被成功接收并处理
3** 重定向,需要进一步的操作以完成请求
4** 客户端错误,请求包含语法错误或无法完成请求
5** 服务器错误,服务器在处理请求的过程中发生了错误

HTTP状态码的详细描述请参考:HTTP状态码列表

网络爬虫基本原理: 采集网络数据的步骤¶

步骤 工具
1 获取目标网页 requests
2 解析html文件提取有用信息 BeautifulSoup, re
3 保存数据 csv, txt, json
  • 采集网络数据:手动采集vs计算机程序采集
  • 学习网络爬虫:上帝视角vs探索者视角

Python爬虫工具:Requests¶

Requests is an elegant and simple HTTP library for Python, built for human beings.

Python爬虫工具:BeautifulSoup¶

Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你节省数小时甚至数天的工作时间.

Python爬虫工具: 正则表达式¶

本页内容取自Jake Vander Plas的 Whirlwind Tour of Python;相关资料可从GitHub获取。Python正则表达式的完整介绍请参考技术文档。

Character Description Example
\d (\D) Match any digit (non-digit)
\s (\S) Match any whitespace (non-whitespace)
\w (\W) Match any alphanumeric char (non-alphanumeric char)
? Match zero or one repetitions of preceding "ab?" matches "a" or "ab"
* Match zero or more repetitions of preceding "ab*" matches "a", "ab", "abb", "abbb"...
+ Match one or more repetitions of preceding "ab+" matches "ab", "abb", "abbb"... but not "a"
{n} Match n repetitions of preeeding "ab{2}" matches "abb"
{m,n} Match between m and n repetitions of preceding "ab{2,3}" matches "abb" or "abbb"
BTC爬取区块数据¶
  • 数据来源:区块链浏览器(Blockchain.info)
  • 获取数据的方式/链接:

    • https://blockchain.info/rawblock/$block_hash

    • https://blockchain.info/rawblock/$block_hash?format=hex

    • https://blockchain.info/block-height/$block_height?format=json

In [ ]:
import requests
import warnings
warnings.filterwarnings('ignore')
In [ ]:
url='https://blockchain.info/block-height/00000?format=json'
r=requests.get(url)
In [ ]:
#print(r.text)
j=r.json()
j
Out[ ]:
{'blocks': [{'hash': '000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f',
   'ver': 1,
   'prev_block': '0000000000000000000000000000000000000000000000000000000000000000',
   'mrkl_root': '4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b',
   'time': 1231006505,
   'bits': 486604799,
   'next_block': ['00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048'],
   'fee': 0,
   'nonce': 2083236893,
   'n_tx': 1,
   'size': 285,
   'block_index': 0,
   'main_chain': True,
   'height': 0,
   'weight': 1140,
   'tx': [{'hash': '4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b',
     'ver': 1,
     'vin_sz': 1,
     'vout_sz': 1,
     'size': 204,
     'weight': 816,
     'fee': 0,
     'relayed_by': '0.0.0.0',
     'lock_time': 0,
     'tx_index': 2098408272645986,
     'double_spend': False,
     'time': 1231006505,
     'block_index': 0,
     'block_height': 0,
     'inputs': [{'sequence': 4294967295,
       'witness': '',
       'script': '04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73',
       'index': 0,
       'prev_out': {'n': 4294967295,
        'script': '',
        'spending_outpoints': [{'n': 0, 'tx_index': 2098408272645986}],
        'spent': True,
        'tx_index': 0,
        'type': 0,
        'value': 0}}],
     'out': [{'type': 0,
       'spent': False,
       'value': 5000000000,
       'spending_outpoints': [],
       'n': 0,
       'tx_index': 2098408272645986,
       'script': '4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac',
       'addr': '1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa'}]}]}]}
In [ ]:
print("dtype: %s, length: %d" %(type(j),len(j)))
dtype: <class 'dict'>, length: 1
In [ ]:
blocks=j['blocks'][0]
print(type(blocks))
blocks.keys() 
<class 'dict'>
Out[ ]:
dict_keys(['hash', 'ver', 'prev_block', 'mrkl_root', 'time', 'bits', 'next_block', 'fee', 'nonce', 'n_tx', 'size', 'block_index', 'main_chain', 'height', 'weight', 'tx'])
In [ ]:
tx=blocks['tx']
print(len(tx))
tx
1
Out[ ]:
[{'hash': '4a5e1e4baab89f3a32518a88c31bc87f618f76673e2cc77ab2127b7afdeda33b',
  'ver': 1,
  'vin_sz': 1,
  'vout_sz': 1,
  'size': 204,
  'weight': 816,
  'fee': 0,
  'relayed_by': '0.0.0.0',
  'lock_time': 0,
  'tx_index': 2098408272645986,
  'double_spend': False,
  'time': 1231006505,
  'block_index': 0,
  'block_height': 0,
  'inputs': [{'sequence': 4294967295,
    'witness': '',
    'script': '04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73',
    'index': 0,
    'prev_out': {'n': 4294967295,
     'script': '',
     'spending_outpoints': [{'n': 0, 'tx_index': 2098408272645986}],
     'spent': True,
     'tx_index': 0,
     'type': 0,
     'value': 0}}],
  'out': [{'type': 0,
    'spent': False,
    'value': 5000000000,
    'spending_outpoints': [],
    'n': 0,
    'tx_index': 2098408272645986,
    'script': '4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac',
    'addr': '1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa'}]}]

爬取前10个BTC区块的信息¶

待爬取信息¶

  1. 区块高度:height
  2. 区块hash:hash
  3. 交易数量: n_tx
  4. 时间:time

信息来源¶

https://blockchain.info/block-height/$block_height?format=json

存储格式¶

  • csv
In [ ]:
import requests
import csv

with open('./btc_block_info.csv','w',newline='') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['height','hash','n_tx','time'])
    for i in range(10):
        r=requests.get('https://blockchain.info/block-height/0000'+str(i)+'?format=json')
        blocks=r.json()['blocks'][0]
        writer.writerow([i,blocks['hash'],blocks['n_tx'],blocks['time']]) 
        print('i=',i,' completed')
i= 0  completed
i= 1  completed
i= 2  completed
i= 3  completed
i= 4  completed
i= 5  completed
i= 6  completed
i= 7  completed
i= 8  completed
i= 9  completed
In [ ]:
[i,blocks['hash'],blocks['n_tx'],blocks['time']]
Out[ ]:
[9,
 '000000008d9dc510f23c2657fc4f67bea30078cc05a90eb89e84cc475c080805',
 1,
 1231473279]

3. 使用API¶

  1. #### get_block Get a single block based on a block hash. Returns a Block object.

Params:

block_id : str - block hash

Usage:

from blockchain import blockexplorer

block = blockexplorer.get_block('000000000000000016f9a2c3e0f4c1245ff24856a79c34806969f5084f410680')
In [ ]:
from blockchain import blockexplorer
block = blockexplorer.get_block('000000000000000016f9a2c3e0f4c1245ff24856a79c34806969f5084f410680')
In [ ]:
block.n_tx
Out[ ]:
345
  1. #### get_block_height Get an array of blocks at the specified height. Returns an array of Block objects.

Params:

height : int - block height

Usage:

blocks = blockexplorer.get_block_height(2570)
In [ ]:
blocks = blockexplorer.get_block_height(2570)
In [ ]:
print(blocks[0].merkle_root)
blocks[0].n_tx
115f940a1363ca9ab303fdd3fdddb8d6d0d98fbfc013419fc66917b2bded5208
Out[ ]:
1
  1. #### get_blocks Get a list of blocks for a specific day or mining pool. Returns an array of SimpleBlock objects.

Params:

time : int - unix time in ms (optional)
pool_name : str - pool name (optional)

At least one parameter is required.

Usage:

blocks = blockexplorer.get_blocks(pool_name = 'Discus Fish')

Response object field definitions¶

Block¶

hash : str
version : int
previous_block : str
merkle_root : str
time : int
bits : int
fee : int
nonce int
n_tx : int
size : int
block_index : int
main_chain : bool
height : int
received_time : int
relayed_by : string
transactions : array of Transaction objects

Transaction¶

double_spend : bool
block_height : int (if -1, the tx is unconfirmed)
time : int
relayed_by : str
hash : str
tx_index : int
version : int
size : int
inputs : array of Input objects
outputs: array of Output objects

Input¶

n : int
value : int
address : str
tx_index : int
type : int
script : str
script_sig : str
sequence : int

Note: if coinbase transaction, then only script and script_siq will be populated.

Output¶

n : int
value : int
address : str
tx_index : int
script : str
spent : bool

Address¶

hash160 : str
address : str
n_tx : int
total_received : int
total_sent : int
final_balance : int
transactions : array of Transaction objects

MultiAddress¶

n_tx : int
n_tx_filtered : int
total_received : int
total_sent : int
final_balance : int
addresses : array of SimpleAddress objects
transactions : array of Transaction objects

Balance¶

n_tx : int
total_received : int
final_balance : int

UnspentOutput¶

tx_hash : str
tx_index : int
tx_output_n : int
script : str
value : int
value_hex : str
confirmations : int

LatestBlock¶

hash : str
time : int
block_index : int
height : int
tx_indexes : array of TX indexes (integers)

SimpleBlock¶

height : int
hash : str
time : int
main_chain : bool

区块链技术的应用
¶

习近平在中央政治局第十八次集体学习时强调
把区块链作为核心技术自主创新重要突破口 加快推动区块链技术和产业创新发展

2019-10-25 18:14:26 来源: 新华网

        新华社北京10月25日电 中共中央政治局10月24日下午就区块链技术发展现状和趋势进行第十八次集体学习。中共中央总书记习近平在主持学习时强调,区块链技术的集成应用在新的技术革新和产业变革中起着重要作用。我们要把区块链作为核心技术自主创新的重要突破口,明确主攻方向,加大投入力度,着力攻克一批关键核心技术,加快推动区块链技术和产业创新发展。

  浙江大学教授、中国工程院院士陈纯就这个问题作了讲解,并谈了意见和建议。

  中共中央政治局各位同志认真听取了讲解,并进行了讨论。

        习近平在主持学习时发表了讲话。他指出,区块链技术应用已延伸到**数字金融、物联网、智能制造、供应链管理、数字资产交易**等多个领域。目前,全球主要国家都在加快布局区块链技术发展。我国在区块链领域拥有良好基础,要加快推动区块链技术和产业创新发展,积极推进区块链和经济社会融合发展。

  习近平强调,要强化基础研究,提升原始创新能力,努力让我国在区块链这个新兴领域走在理论最前沿、占据创新制高点、取得产业新优势。要推动协同攻关,加快推进核心技术突破,为区块链应用发展提供安全可控的技术支撑。要加强区块链标准化研究,提升国际话语权和规则制定权。要加快产业发展,发挥好市场优势,进一步打通创新链、应用链、价值链。要构建区块链产业生态,**加快区块链和人工智能、大数据、物联网等前沿信息技术的深度融合**,推动集成创新和融合应用。要加强人才队伍建设,建立完善人才培养体系,打造多种形式的高层次人才培养平台,培育一批领军人物和高水平创新团队。

        习近平指出,要抓住区块链技术融合、功能拓展、产业细分的契机,发挥区块链在**促进数据共享、优化业务流程、降低运营成本、提升协同效率、建设可信体系等方面的作用**。要推动区块链和实体经济深度融合,**解决中小企业贷款融资难、银行风控难、部门监管难**等问题。要利用区块链技术探索数字经济模式创新,为打造便捷高效、公平竞争、稳定透明的营商环境提供动力,为推进供给侧结构性改革、实现各行业供需有效对接提供服务,为加快新旧动能接续转换、推动经济高质量发展提供支撑。要探索“区块链+”在民生领域的运用,积极推动区块链技术在**教育、就业、养老、精准脱贫、医疗健康、商品防伪、食品安全、公益、社会救助等领域的应用**,为人民群众提供更加智能、更加便捷、更加优质的公共服务。要推动区块链底层技术服务和新型智慧城市建设相结合,**探索在信息基础设施、智慧交通、能源电力等领域**的推广应用,提升城市管理的智能化、精准化水平。要利用区块链技术**促进城市间在信息、资金、人才、征信等方面更大规模的互联互通**,保障生产要素在区域内有序高效流动。要探索利用区块链数据共享模式,*实现政务数据跨部门、跨区域共同维护和利用**,促进业务协同办理,深化“最多跑一次”改革,为人民群众带来更好的政务服务体验。

        习近平强调,要加强对区块链技术的引导和规范,加强对区块链安全风险的研究和分析,密切跟踪发展动态,积极探索发展规律。要探索建立适应区块链技术机制的安全保障体系,引导和推动区块链开发者、平台运营者加强行业自律、落实安全责任。要把依法治网落实到区块链管理中,推动区块链安全有序发展。

  习近平指出,相关部门及其负责领导同志要注意区块链技术发展现状和趋势,提高运用和管理区块链技术能力,使区块链技术在建设网络强国、发展数字经济、助力经济社会发展等方面发挥更大作用。

Components of a Blockchain¶

The components of an open, public blockchain are (usually):

  • A peer-to-peer (P2P) network connecting participants and propagating transactions and blocks of verified transactions, based on a standardized "gossip" protocol

  • Messages, in the form of transactions, representing state transitions

  • A set of consensus rules, governing what constitutes a transaction and what makes for a valid state transition

  • A state machine that processes transactions according to the consensus rules

  • A chain of cryptographically secured blocks that acts as a journal of all the verified and accepted state transitions

  • A consensus algorithm that decentralizes control over the blockchain, by forcing participants to cooperate in the enforcement of the consensus rules

  • A game-theoretically sound incentivization scheme (e.g., proof-of-work costs plus block rewards) to economically secure the state machine in an open environment

  • One or more open source software implementations of the above ("clients")

DLT平台¶

BaaS¶

(An incomplete list) Reference (from top finance journals)¶

[1] Liu, Yukun, Aleh Tsyvinski, and Xi Wu. "Common risk factors in cryptocurrency." The Journal of Finance 77.2 (2022): 1133-1177.

[2] Griffin, John M., and Amin Shams. "Is Bitcoin really untethered?." The Journal of Finance 75.4 (2020): 1913-1964.

[3] Hinzen, Franz J., Kose John, and Fahad Saleh. "Bitcoin’s limited adoption problem." Journal of Financial Economics 144.2 (2022): 347-369.

[4] Easley, David, Maureen O'Hara, and Soumya Basu. "From mining to markets: The evolution of bitcoin transaction fees." Journal of Financial Economics 134.1 (2019): 91-109.

[5] Makarov, Igor, and Antoinette Schoar. "Trading and arbitrage in cryptocurrency markets." Journal of Financial Economics 135.2 (2020): 293-319.

[6] Sokolov, Konstantin. "Ransomware activity and blockchain congestion." Journal of Financial Economics 141.2 (2021): 771-782.

[7] Cong, Lin William, Ye Li, and Neng Wang. "Token-based platform finance." Journal of Financial Economics (2021).

[8] Gryglewicz, Sebastian, Simon Mayer, and Erwan Morellec. "Optimal financing with tokens." Journal of Financial Economics 142.3 (2021): 1038-1067.

[9] Howell, Sabrina T., Marina Niessner, and David Yermack. "Initial coin offerings: Financing growth with cryptocurrency token sales." The Review of Financial Studies 33.9 (2020): 3925-3974.

[10] Liu, Yukun, and Aleh Tsyvinski. "Risks and returns of cryptocurrency." The Review of Financial Studies 34.6 (2021): 2689-2727.

[11] Foley, Sean, Jonathan R. Karlsen, and Tālis J. Putniņš. "Sex, drugs, and bitcoin: How much illegal activity is financed through cryptocurrencies?." The Review of Financial Studies 32.5 (2019): 1798-1853.

[12] Cong, Lin William, Ye Li, and Neng Wang. "Tokenomics: Dynamic adoption and valuation." The Review of Financial Studies 34.3 (2021): 1105-1155.

[13] Pagnotta, Emiliano S. "Decentralizing money: Bitcoin prices and blockchain security." The Review of Financial Studies 35.2 (2022): 866-907.

[14] Biais, Bruno, et al. "The blockchain folk theorem." The Review of Financial Studies 32.5 (2019): 1662-1715.

[15] Cong, Lin William, Zhiguo He, and Jiasun Li. "Decentralized mining in centralized pools." The Review of Financial Studies 34.3 (2021): 1191-1235.

[16] Goldstein, Itay, Wei Jiang, and G. Andrew Karolyi. "To FinTech and beyond." The Review of Financial Studies 32.5 (2019): 1647-1661.

[17] Saleh, Fahad. "Blockchain without waste: Proof-of-stake." The Review of financial studies 34.3 (2021): 1156-1190.

[18] Chiu, Jonathan, and Thorsten V. Koeppl. "Blockchain-based settlement for asset trading." The Review of Financial Studies 32.5 (2019): 1716-1753.

[19] Cong, Lin William, and Zhiguo He. "Blockchain disruption and smart contracts." The Review of Financial Studies 32.5 (2019): 1754-1797.

[20] Lee, Jeongmin, and Christine A. Parlour. "Consumers as financiers: Consumer surplus, crowdfunding, and Initial Coin Offerings." The Review of Financial Studies 35.3 (2022): 1105-1140.

[21] Chen, Mark A., Qinxi Wu, and Baozhong Yang. "How valuable is FinTech innovation?." The Review of Financial Studies 32.5 (2019): 2062-2106.

[22] Cheng, Stephanie F., et al. "Riding the blockchain mania: Public firms’ speculative 8-K disclosures." Management Science 65.12 (2019): 5901-5913.

[23] Whitaker, Amy, and Roman Kräussl. "Fractional equity, blockchain, and the future of creative work." Management Science 66.10 (2020): 4594-4611.

[24] Holden, Richard, and Anup Malani. "An Examination of Velocity and Initial Coin Offerings." Management Science (2022).

[25] Tsoukalas, Gerry, and Brett Hemenway Falk. "Token-weighted crowdsourcing." Management Science 66.9 (2020): 3843-3859.

[26] Gan, Jingxing, Gerry Tsoukalas, and Serguei Netessine. "Initial coin offerings, speculation, and asset tokenization." Management Science 67.2 (2021): 914-931.

[27] Malik, Nikhil, et al. "Why Bitcoin will Fail to Scale?." Management Science (2022).

[28] Roşu, Ioanid, and Fahad Saleh. "Evolution of shares in a proof-of-stake cryptocurrency." Management Science 67.2 (2021): 661-672.

[29] Csóka, Péter, and P. Jean-Jacques Herings. "Decentralized clearing in financial networks." Management Science 64.10 (2018): 4681-4699.

[30] Lyandres, Evgeny, Berardino Palazzo, and Daniel Rabetti. "Initial coin offering (ico) success and post-ico performance." Management Science (2022).

[31] Arnosti, Nick, and S. Matthew Weinberg. "Bitcoin: A natural oligopoly." Management Science (2022).