代理人模型笔记 - ABM在现代经济的应用暨外汇量化、新闻政史交易实践¶
这个笔记的是笔者的学习手记，内容包括统计、建模（EBM、ABM）、数据处理平台、外汇信号系统、量化交易、机器人制作等内容，较为繁杂。读者根据自己的需求酌情阅读。（意思就是你通读了这本书并不会对你产生太大影响，但是如果能改变你的一点点思想，那就是好的。)
笔者的系统理论来主要自于自控理论，建模技能偏向于ABM建模，然后是数学建模，最后的应用点，也就是本文的计算机与交易系统。
随着学习的深入，第二章内容需要进一步扩展，因而agent本身已经不够，需要更强的能力。
其中会牵涉到很多时下的热点，比如大数据、云计算、工业4.0、物联网等，计算机工具在量化投资中其实早已广泛使用，尤其超算、机器学习、高速网络等尖端技术。嗯，叫法而已，它们细化到产品并落地到市场上后就是个工具。当然，对这些工具的专业掌握程度决定着产品的质量，但就本文而言，知其然与所以然即可，无需深究，除非遇到瓶颈（发动机制造商只要知道选用并采购什么样的阀门、控制器等材料组件就行了，不需要每一样东西都自己生产）。
最终，这些所有东西都可以以服务形式包装提供给用户，从而形成XX云，比如这个平台最终形成的代理人建模云。。（不好听）
总共分为三个部分，第一个部分是基础知识，包括统计、算法、模型、工具；第二个部分是外汇实践，包括基本分析、技术分析、新闻分析、舆论分析、信号系统。
最后，判断你知识是否牢固的标准就是这个系统的盈利成长，多简单，多直接，多赤裸。当然，你赚的少有时并不是你知识真的不牢固，而是你的模型仍需优化。
这些知识可能走了很多弯路，但，游戏本身就是用来探索的，冯·诺伊曼还用无穷级数算苍蝇飞多远呢。还有就是现行的游戏理论在强劲的计算能力面前显得不那么合适，比如他们以国际象棋举例时，肯定不知道现在计算机拥有几乎全部棋谱与策略，人类就这点战胜计算机异常困难，这也可能也适用于正在流行的围棋。
与这个笔记对应的网站是https://forex.fusionworks.cn，里面有一些个人交易资料。
哦，还有个公众号，vola！
UPDATE 2020-07
好久没更新了
这里是主架构图。
第一章 数据收集、统计¶
在开始这一章之前，你可能需要补习一下数学知识；还有熟悉下常见工具（语言），不必多年开发经验，会处理常见数据结构、能格式化文件即可。
建议先通读一下 Scrapy 中文文档 ，这样你会省去好多Google的时间；在 知乎 上有许多关于 大数据 、 数据挖掘 的讨论，你可以去看看了解一些业内的动态。
另外，可以使用 Nutch 来爬取，并用 Solr 来构建一个简单的搜索引擎，它们可以跟下一章节的Hadoop集成。
还有一个比较重要的点-- Model Thinking ，你需要的不只是建模的知识，还要有建模的思想。数据和算法并不是最重要的，重要的是你如何利用这些数据通过你设计的模型来输出对你有用的结果。
不要以编程开始你的机器学习之旅，这样容易使思维受限于语言，通过对模型和结果的思考达到你的目的，编程只是手段之一。 话虽如此，但是你必须学会的语言有Python、NetLogo、MQL，额外的语言为Bash、C、C++，如果你会点Java那就更好啦。如何学习它们？OK，下载手册，那些命令与类型一个个敲着学，不要漏过任何一个。R语言的效率可能并不是很好，工程类喜欢Python，科学家喜欢S-Plus，R也流行。
建模工具这块儿，笔者认为最适合个人学习的当属Netlogo，其他开源项目有可直接用 GAMA 或者 Mesa ，也有商业产品比如IBM SPSS、AnyLogic系列，模拟股市的也有 Altreva Adaptive Modeler 、QLearning Trading，但是。。。扩展性支持上Netlogo灵活而多样，如果需要超算版本可以也可以试试Repast。
NetLogo: netlogo_help.7z
如果读者已经具备了以下几点的基本知识，那么，跳过这章吧。
1.1 数据收集¶
数据收集是学习数据分析的开始。
为了省去一些学习的麻烦，我找了一些 “大”数据 ，有些上百TB的数据对非行业内的人来说可能毫无意义，但是，先来些数据吧，它们对学习者来说还是比较实用的。
1.2 爬虫示例¶
58同城¶
我简单写了一个 收集58同城中上海出租房信息的爬虫 ，包括的条目有： 描述 、 位置 、 价格 、 房间数 、 URL 。
由于这些信息都可以在地图上表示出来，那我除了画统计图以外还会画它们在地图上的表示。
1.3 统计学关键字¶
第二章 机器学习基础¶
2.1 numpy 快查¶
import numpy as np
a = np.arange(1,5)
data_type = [('name','S10'), ('height', 'float'), ('age', int)]
values = [('Arthur', 1.8, 41), ('Lancelot', 1.9, 38),
('Galahad', 1.7, 38)]
b = np.array(values, dtype=data_type)
c = np.arange(6,10)
# 符号
np.sign(a)
# 数组最大值
a.max()
# 数组最小值
a.max()
# 区间峰峰值
a.ptp()
# 乘积
a.prod()
# 累积
a.cumprod()
# 平均值
a.mean()
# 中值
a.median()
# 差分
np.diff(a)
# 方差
np.var(a)
# 元素条件查找，返回index的array
np.where(a>2)
# 返回第2，3，5个元素的array
np.take(a, np.array(1,2,4))
# 排序
np.msort(a)
np.sort(b, kind='mergesort', order='height')
# 均分，奇数个元素的array不可分割为偶数。
np.split(b,2)
# 创建单位矩阵
np.eye(3)
# 最小二乘，参数为[x,y,degree]，degree为多项式的最高次幂，返回值为所有次幂的系数
np.polyfit(a,c,1)
2.2 监督学习¶
信息分类基础¶
信息的不稳定性为熵（entropy），而信息增益为有无样本特征对分类问题影响的大小。比如，抛硬币正反两面各有50%概率，此时不稳定性最大，熵为1；太阳明天照常升起，则是必然，此事不稳定性最小，熵为0。
假设事件X，发生概率为x，其信息期望值定义为：
整个信息的熵为：
如何找到最好的分类特征：
def chooseBestFeatureToSplit(dataSet):
numFeatures = len(dataSet[0]) - 1 #the last column is used for the labels
baseEntropy = calcShannonEnt(dataSet)
bestInfoGain = 0.0; bestFeature = -1
for i in range(numFeatures): #iterate over all the features
featList = [example[i] for example in dataSet]#create a list of all the examples of this feature
uniqueVals = set(featList) #get a set of unique values
newEntropy = 0.0
for value in uniqueVals:
subDataSet = splitDataSet(dataSet, i, value)
prob = len(subDataSet)/float(len(dataSet))
newEntropy += prob * calcShannonEnt(subDataSet)
infoGain = baseEntropy - newEntropy #calculate the info gain; ie reduction in entropy
if (infoGain > bestInfoGain): #compare this to the best gain so far
bestInfoGain = infoGain #if better than current best, set to best
bestFeature = i
return bestFeature #returns an integer
其中，dataSet为所有特征向量，caclShannonEnt()计算特征向量的熵，splitDataSet()切除向量中的value列；infoGain即为信息增益，chooseBestFeatureToSplit返回最好的特征向量索引值。
K邻近算法¶
kNN的算法模型如下：
对于未知类别属性的数据且集中的每个点依次执行以下操作：
- 计算已知类别数据集中的点与当前点之间的距离
- 按照距离递增依次排序
- 选取与当前点距离最小的k个点
- 确定前k个点所在类别的出现频率
- 返回前k个点出现频率最高的类别作为当前点的预测分类
代码参考如下：
def classify0(inX, dataSet, labels, k):
dataSetSize = dataSet.shape[0]
diffMat = tile(inX, (dataSetSize,1)) - dataSet
sqDiffMat = diffMat**2
sqDistances = sqDiffMat.sum(axis=1)
distances = sqDistances**0.5
sortedDistIndicies = distances.argsort()
classCount={}
for i in range(k):
voteIlabel = labels[sortedDistIndicies[i]]
classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
sortedClassCount = sorted(classCount.iteritems(), key=operator.itemgetter(1), reverse=True)
return sortedClassCount[0][0]
其中，inX为输入向量，dataSet为数据集，labels为数据集的分类，可调。距离计算公式为d0 = ((x-x0)**2 + (y-y0)**2)**0.5。
此种算法的优点为精度高、对异常值不敏感、但缺点也比较明显，即数据量大时开支相对较大，适用于数值－标称型数据。
决策树¶
决策树即列出一系列选择，根据训练集中的大量形似（A、B、C）以及结果D的向量来预测新输入（A'、B'、C'）的结果D'。
首先创建一个决策树：
def createTree(dataSet,labels):
classList = [example[-1] for example in dataSet]
if classList.count(classList[0]) == len(classList):
return classList[0] #stop splitting when all of the classes are equal
if len(dataSet[0]) == 1: #stop splitting when there are no more features in dataSet
return majorityCnt(classList)
bestFeat = chooseBestFeatureToSplit(dataSet)
bestFeatLabel = labels[bestFeat]
myTree = {bestFeatLabel:{}}
del(labels[bestFeat])
featValues = [example[bestFeat] for example in dataSet]
uniqueVals = set(featValues)
for value in uniqueVals:
subLabels = labels[:] #copy all of labels, so trees don't mess up existing labels
myTree[bestFeatLabel][value] = createTree(splitDataSet(dataSet, bestFeat, value),subLabels)
return myTree
找到影响最大的特征bestFeat后，再创建此特征下的分类向量创建子树向量，然后将bestFeat分离后继续迭代，直至所有特征都转换成决策节点。
原始数据比如：
no-surfacing flippers fish
1 yes yes yes
2 yes yes yes
3 yes no no
4 no yes no
5 no yes no
会生成如下决策树：
no-surfacing?
/ \
no/ \yes
fish(no) flippers?
/ \
no/ \yes
fish(no) fish(yes)
表示成JSON格式，即python字典：
{'no surfacing':{0:'no',1:{'flippers':{0:'no',1:'yes'}}}
构建决策树的方法比较多，也可使用C4.5和CART算法。
接下来使用决策树进行分类：
def classify(inputTree,featLabels,testVec):
firstStr = inputTree.keys()[0]
secondDict = inputTree[firstStr]
featIndex = featLabels.index(firstStr)
key = testVec[featIndex]
valueOfFeat = secondDict[key]
if isinstance(valueOfFeat, dict):
classLabel = classify(valueOfFeat, featLabels, testVec)
else: classLabel = valueOfFeat
return classLabel
其中，featLabels为测试的判断节点，即特征，testVec为其值，比如classify[myTree,"['no-surfacing','flippers']",:[1,1]"]，如此结果便为'no'。
使用pickle对决策树进行序列化存储：
def storeTree(inputTree,filename):
import pickle
fw = open(filename,'w')
pickle.dump(inputTree,fw)
fw.close()
其中，dump可选协议为0（ASCII），1（BINARY），默认为0；读取时使用pickle.load；同样可使用dumps，loads直接对字符变量进行操作。
此种算法计算复杂度不高，对中间值缺失不敏感，但可能会产生过拟合的问题。
朴素贝叶斯¶
贝叶斯模型是基于独立概率统计的，思想可以这么说：
总共7个石子在A、B两个桶中，A桶中有2黑2白，B桶中有2黑1白。已知条件为石子来自B桶，那么它是白色石子的概率可表示为：
P(white|B)=P(B|white)P(white)/P(B)
接下来，定义两个事件A、B，P(A|B)与P(B|A)相互转化的过程即为：
P(B|A)=P(A|B)P(B)/P(A)
而朴素贝叶斯可以这样描述：
设x={a1,a2,...,am}为待分类项，a为x的特征属性，类别集合为C={y1,y2,...,ym}，如果P(yk|x)=max(P(y1|x),P(y2|x),...,P(yn|x))，则x属于yk。
整个算法核心即是等式P(yi|x)=P(x|yi)P(yi)/P(x)。
首先构建一个分类训练函数（二元分类）：
def trainNB0(trainMatrix,trainCategory):
numTrainDocs = len(trainMatrix)
numWords = len(trainMatrix[0])
pBad = sum(trainCategory)/float(numTrainDocs)
p0Num = ones(numWords); p1Num = ones(numWords) #change to ones()
p0Denom = 2.0; p1Denom = 2.0 #change to 2.0
for i in range(numTrainDocs):
if trainCategory[i] == 1:
p1Num += trainMatrix[i]
p1Denom += sum(trainMatrix[i])
else:
p0Num += trainMatrix[i]
p0Denom += sum(trainMatrix[i])
p1Vect = log(p1Num/p1Denom) #change to log()
p0Vect = log(p0Num/p0Denom) #change to log()
return p0Vect,p1Vect,pBad
其中，trainMatrix为所有训练集中的布尔向量，比如两本书A、B，其中A有两个单词x、y，B有两个单词x、z，并且A是好书（值计为0），B是烂书（值计为0），把所有单词进行排序后得向量['x','y','z']，则A的Matrix可表示为[1,1,0]，B的为[1,0,1]，所以此函数中的trainMatrix即[[1,1,0],[1,0,1]]，trainCategory为[0,1]。
函数返回的为概率集的向量。
分类函数：
def classifyNB(vec2Classify, p0Vec, p1Vec, pClass1):
p1 = sum(vec2Classify * p1Vec) + log(pClass1) #element-wise mult
p0 = sum(vec2Classify * p0Vec) + log(1.0 - pClass1)
if p1 > p0:
return 1
else:
return 0
vec2Classify即为要分类的向量，形如trainMatrix，随后的三个参数为trainNB0所返回。p1、p0可以理解为期望概率值，比较两者大小即可划分。
测试用例：
def testingNB():
listOPosts,listClasses = loadDataSet()
myVocabList = createVocabList(listOPosts)
trainMat=[]
for postinDoc in listOPosts:
trainMat.append(setOfWords2Vec(myVocabList, postinDoc))
p0V,p1V,pAb = trainNB0(array(trainMat),array(listClasses))
testEntry = ['love', 'my', 'dalmation']
thisDoc = array(setOfWords2Vec(myVocabList, testEntry))
print testEntry,'classified as: ',classifyNB(thisDoc,p0V,p1V,pAb)
testEntry = ['stupid', 'garbage']
thisDoc = array(setOfWords2Vec(myVocabList, testEntry))
print testEntry,'classified as: ',classifyNB(thisDoc,p0V,p1V,pAb)
整体来说，朴素贝叶斯分类方法在数据较少的情况下仍然有效，但是对数据输入比较敏感。
Logistic回归¶
在统计学中，线性回归是利用称为线性回归方程的最小二乘函数对一个或多个自变量和因变量之间关系进行建模的一种回归分析。这种函数是一个或多个称为回归系数的模型参数的线性组合。只有一个自变量的情况称为简单回归，大于一个自变量情况的叫做多元回归。（ 维基百科 ）
先介绍两个重要的数学概念。
最小二乘法则
最小二乘法（又称最小平方法）是一种数学优化技术。它通过最小化误差的平方和寻找数据的最佳函数匹配。
利用最小二乘法可以简便地求得未知的数据，并使得这些求得的数据与实际数据之间误差的平方和为最小。
示例1
有四个数据点(1,6)、(2,5)、(3,7)、(4,10)，我们希望找到一条直线y=a+bx与这四个点最匹配。
采用最小二乘法使等号两边的方差尽可能小，也就是找出这个函数的最小值：
然后对S(a,b)求a,b的偏导数，使其为0得到：
这样就解出：
所以直线y=3.5+1.4x是最佳的。
函数表示
欧几里德表示
线性函数模型
典型的一类函数模型是线性函数模型。最简单的线性式是
写成矩阵式，为
直接给出该式的参数解：
其中
为t值的算术平均值。也可解得如下形式：
示例2
随机选定10艘战舰，并分析它们的长度与宽度，寻找它们长度与宽度之间的关系。由下面的描点图可以直观地看出，一艘战舰的长度（t）与宽度（y）基本呈线性关系。散点图如下：
以下图表列出了各战舰的数据，随后步骤是采用最小二乘法确定两变量间的线性关系。
仿照上面给出的例子
并得到相应的
然后确定b1
可以看出，战舰的长度每变化1m，相对应的宽度便要变化16cm。并由下式得到常数项b0：
可以看出点的拟合非常好，长度和宽度的相关性大约为96.03％。 利用Matlab得到拟合直线：
Sigmoid函数
Sigmoid函数具有单位阶跃函数的性质，公式表示为：
我们将输入记为z，有下面的公式得出：
使用向量写法：
其中向量x是分类器的输入数据，向量w就是我们要找到的最佳系数。
基于优化方法确定回归系数
梯度上升/下降法
梯度上升法/下降法的思想是：要找到函数的最大值，最好的方法是沿着该函数的梯度方向探寻，函数f(x,y)的梯度如下表示：
可以这样理解此算法：
从前有一座山，一个懒人要爬山，他从山脚下的任意位置向山顶出发，并且知道等高线图的每个环上都有一个宿营点，他希望在这些宿营点之间修建一条笔直的路，并且路到两旁的宿营点的垂直距离差的平方和尽可能小。每到一个等高线圈，他都会根据他在上一个等高线的距离的变化量来调节他的在等高线上的位置，从而使公路满足要求。
返回回归系数：
def sigmoid(x):
return 1.0/(1+math.exp(-x))
def gradAscent(dataMatIn, classLabels):
dataMatrix = mat(dataMatIn) #convert to NumPy matrix
labelMat = mat(classLabels).transpose() #convert to NumPy matrix
m,n = shape(dataMatrix)
alpha = 0.001
maxCycles = 500
weights = ones((n,1))
for k in range(maxCycles): #heavy on matrix operations
h = sigmoid(dataMatrix*weights) #matrix mult
error = (labelMat - h) #vector subtraction
weights = weights + alpha * dataMatrix.transpose()* error #matrix mult
return weights
其中，误差值乘以矩阵的转秩代表梯度。
接下来，我们载入数据集尝试画出最佳拟合直线：
import matplotlib.pyplot as plt
def plotBestFit(w):
weights = w.getA() # 将矩阵转化为数组
dataMat, labelMat = loadDataSet()
dataArr = array(dataMat)
n = shape(dataArr)[0]
x1 = x2 = x3 = x4 = []
for i in range(n):
if int(labelMat[i] == 1:
x1.append(dataArr[i,1]); y1.append(dataArr[i,2])
else
x2.append(dataArr[i,1]); y2.append(dataArr[i,2])
fig = plt.figure()
ax = fig.add_subplot(111)
ax.scatter(x1, y1, s=30, c='red', markers='s')
ax.scatter(x2, y2, s=30, c='green')
x = arrange(-3.0, 3.0, 0.1)
y = (-weight[0]-weights[1]*x)/weights[2]
ax.plot(x, y)
plt.xlabel('X1')
plt.xlabel('X2')
plt.show()
如果处理大量数据集时，可以使用批量数据集中的单个数据点进行系数更新，或者使用随机数据集的数据点，分别如下所示：
def stocGradAscentBatch(dataMatrix, classLabels):
m,n = shape(dataMatrix)
alpha = 0.01
weights = ones(n)
for i in range(m):
h = sigmoid(sum(dataMatrix[i]*weights))
error = classLabels[i] - h
weights = weights + alpha * error * dataMatrix[i]
return weights
def stocGradAscentRand(dataMatrix, classLabels, numIter=150)
m,n = shape(dataMatrix)
weights = ones(n)
for j in range(numIter)
dataIndex = range(m)
for i in range(m):
alpha = 4/(1.0+j+i)+0.01 # 调整
randIndex = int(random.uniform(0, len(dataIndex)))
h = sigmoid(sum(dataMatrix[randIndex]*weights))
error = classLabels[randIndex] - h
weights = weights + alpha * error * dataMatrix[randIndex]
del(dataIndex[randIndex]) # 删除已选，防止重复选取
线性回归¶
树回归¶
SVM¶
SVM（Supprot Vector Machines）即支持向量机，完全理解其理论知识对数学要求较高，以笔者的二半吊子水平不足以完全应付，所以，我就培养下感性认识吧。
以下内容来自 向5岁孩子解释SVM ：
桌子上有两种颜色的球，
我们需要在中间摆一根棍子把它们分开，
完美，那么再加点球，发现再找到一个合适位置的话比较难了（但仍然可以找到），
SVM就是试图把棍放在最佳位置，好让在棍的两边有尽可能大的间隙，
这根棍仍然可以分开它们，
如果，这样摆呢？
你怒拍桌子，将球高高弹起，然后向忍着水果那样在空中划过笔直的一刀。就这样分开了。
那一瞬，球在另一个维度被完美分开了，像这样。
再之后无聊的大人们，把这些球叫做 data，把棍子 叫做 classifier, 最大间隙的把戏叫做optimization， 拍桌子叫做kernelling, 那张纸叫做hyperplane。
以2维数据为例，即平面上的点，可以用直线分割，一般形式为y=ax+b，如果不可分割，我们就将其提高一个维度，分割线变成了分割面（超平面），写作：
神经网络（深度学习）¶
强化学习¶
AdaBoost¶
2.4 数据可视化¶
2.6 显卡的力量¶
第三章 ABM建模基础¶
ABM(Agent Based Modeling)与EBM(Equation Based Modeling)这两种建模方式受不同的人群与场景欢迎。比如在经济学研究中，早期EBM为主，随着ABM的流行有部分人则尝试使用它，但不是说有它就足够了。《ABM in Economy》这本书做了一个很好的示例，ABM结合EBM去研究模型，即EBM作为ABM的理论支撑，使用ABM予以表现模型，从而利于解释与预测一个现象或者模型。
3.1 建模的种类¶
Agent Based Modeling的历史严格来说并不算太久，很多论文中我们都可以看到它是近几十年才成型的建模方法。
3.4 建模思考¶
处理问题的方法。
3.5 模型收集¶
Auction
Auction theory is an applied branch of economics which deals with how people act in auction markets and researches the properties of auction markets. There are many possible designs (or sets of rules) for an auction and typical issues studied by auction theorists include the efficiency of a given auction design, optimal and equilibrium bidding strategies, and revenue comparison. Auction theory is also used as a tool to inform the design of real-world auctions; most notably auctions for the privatization of public-sector companies or the sale of licenses for use of the electromagnetic spectrum.
General idea Auctions are characterized as transactions with a specific set of rules detailing resource allocation according to participants' bids. They are categorized as games with incomplete information because in the vast majority of auctions, one party will possess information related to the transaction that the other party does not (e.g., the bidders usually know their personal valuation of the item, which is unknown to the other bidders and the seller).[1] Auctions take many forms, but they share the characteristic that they are universal and can be used to sell or buy any item. In many cases, the outcome of the auction does not depend on the identity of the bidders (i.e., auctions are anonymous).
Most auctions have the feature that participants submit bids, amounts of money they are willing to pay. Standard auctions require that the winner of the auction is the participant with the highest bid. A nonstandard auction does not require this (e.g., a lottery).
Types of auction Main article: Auction § Types There are traditionally four types of auction that are used for the allocation of a single item:
First-price sealed-bid auctions in which bidders place their bid in a sealed envelope and simultaneously hand them to the auctioneer. The envelopes are opened and the individual with the highest bid wins, paying the amount bid. Second-price sealed-bid auctions (Vickrey auctions) in which bidders place their bid in a sealed envelope and simultaneously hand them to the auctioneer. The envelopes are opened and the individual with the highest bid wins, paying a price equal to the second-highest bid. Open ascending-bid auctions (English auctions) in which participants make increasingly higher bids, each stopping bidding when they are not prepared to pay more than the current highest bid. This continues until no participant is prepared to make a higher bid; the highest bidder wins the auction at the final amount bid. Sometimes the lot is only actually sold if the bidding reaches a reserve price set by the seller. Open descending-bid auctions (Dutch auctions) in which the price is set by the auctioneer at a level sufficiently high to deter all bidders, and is progressively lowered until a bidder is prepared to buy at the current price, winning the auction. Most auction theory revolves around these four "basic" auction types. However, other auction types have also received some academic study (see Auction Types).
Benchmark model The benchmark model for auctions, as defined by McAfee and McMillan (1987), offers a generalization of auction formats, and is based on four assumptions:
All of the bidders are risk-neutral. Each bidder has a private valuation for the item independently drawn from some probability distribution. The bidders possess symmetric information. The payment is represented as a function of only the bids. The benchmark model is often used in tandem with the Revelation Principle, which states that each of the basic auction types is structured such that each bidder has incentive to report their valuation honestly. The two are primarily used by sellers to determine the auction type that maximizes the expected price. This optimal auction format is defined such that the item will be offered to the bidder with the highest valuation at a price equal to their valuation, but the seller will refuse to sell the item if they expect that all of the bidders' valuations of the item are less than their own.[1]
Relaxing each of the four main assumptions of the benchmark model yields auction formats with unique characteristics:
Risk-averse bidders incur some kind of cost from participating in risky behaviors, which affects their valuation of a product. In sealed-bid first-price auctions, risk-averse bidders are more willing to bid more to increase their probability of winning, which, in turn, increases their expected utility. This allows sealed-bid first-price auctions to produce higher expected revenue than English and sealed-bid second-price auctions. In formats with correlated values—where the bidders’ values for the item are not independent—one of the bidders perceiving their value of the item to be high makes it more likely that the other bidders will perceive their own values to be high. A notable example of this instance is the Winner’s curse, where the results of the auction convey to the winner that everyone else estimated the value of the item to be less than they did. Additionally, the linkage principle allows revenue comparisons amongst a fairly general class of auctions with interdependence between bidders' values. The asymmetric model assumes that bidders are separated into two classes that draw valuations from different distributions (i.e., dealers and collectors in an antiques auction). In formats with royalties or incentive payments, the seller incorporates additional factors, especially those that affect the true value of the item (e.g., supply, production costs, and royalty payments), into the price function.[1] Game-theoretic models A game-theoretic auction model is a mathematical game represented by a set of players, a set of actions (strategies) available to each player, and a payoff vector corresponding to each combination of strategies. Generally, the players are the buyer(s) and the seller(s). The action set of each player is a set of bid functions or reservation prices (reserves). Each bid function maps the player's value (in the case of a buyer) or cost (in the case of a seller) to a bid price. The payoff of each player under a combination of strategies is the expected utility (or expected profit) of that player under that combination of strategies.
Game-theoretic models of auctions and strategic bidding generally fall into either of the following two categories. In a private value model, each participant (bidder) assumes that each of the competing bidders obtains a random private value from a probability distribution. In a common value model, the participants have equal valuations of the item, but they do not have perfectly accurate information about this valuation. In lieu of knowing the exact valuation of the item, each participant can assume that any other participant obtains a random signal, which can be used to estimate the true valuation, from a probability distribution common to all bidders.[2] Usually, but not always, a private values model assumes that the values are independent across bidders, whereas a common value model usually assumes that the values are independent up to the common parameters of the probability distribution.
A more general category for strategic bidding is the affiliated values model, in which the bidder's total utility depends on both their individual private signal and some unknown common value. Both the private value and common value models can be perceived as extensions of the general affiliated values model.[3]
Ex-post equilibrium in a simple auction market. When it is necessary to make explicit assumptions about bidders' value distributions, most of the published research assumes symmetric bidders. This means that the probability distribution from which the bidders obtain their values (or signals) is identical across bidders. In a private values model which assumes independence, symmetry implies that the bidders' values are independently and identically distributed (i.i.d.).
An important example (which does not assume independence) is Milgrom and Weber's "general symmetric model" (1982).[4][5] One of the earlier published theoretical research addressing properties of auctions among asymmetric bidders is Keith Waehrer's 1999 article.[6] Later published research include Susan Athey's 2001 Econometrica article,[7] as well as Reny and Zamir (2004).[8]
The first formal analysis of auctions was by William Vickrey (1961). Vickrey considers two buyers bidding for a single item. Each buyer's value, v, is an independent draw from a uniform distribution with support [0,1]. Vickrey showed that in the sealed first-price auction it is an equilibrium bidding strategy for each bidder to bid half his valuation. With more bidders, all drawing a value from the same uniform distribution it is easy to show that the symmetric equilibrium bidding strategy is
Automata
Automata(or Automaton), is a self-operating machine, or a machine or control mechanism designed to automatically follow a predetermined sequence of operations, or respond to predetermined instructions. Some automata, such as bellstrikers in mechanical clocks, are designed to give the illusion to the casual observer that they are operating under their own power.
One of the most famous is "A new kind of science" by Stephen Wolfram(founder of Wolfram Research Inc.). The writer researched lots patterns of cellular automata.
Bell Curves
The Bell Curve: Intelligence and Class Structure in American Life is a 1994 book by psychologist Richard J. Herrnstein and political scientist Charles Murray, in which the authors argue that human intelligence is substantially influenced by both inherited and environmental factors and that it is a better predictor of many personal dynamics, including financial income, job performance, birth out of wedlock, and involvement in crime than are an individual's parental socioeconomic status. They also argue that those with high intelligence, the "cognitive elite", are becoming separated from those of average and below-average intelligence. The book was controversial, especially where the authors wrote about racial differences in intelligence and discussed the implications of those differences.
Shortly after its publication, many people rallied both in criticism and defense of the book. A number of critical texts were written in response to it.
Collective Coorperation
DIKW
Entropy
Fisher
Large Event
Linear
Long tails
Lyapunov
Marknov
Miller Page
Nash Equilibrium
Networks
Percolation
Polya Balancing Process
Prisoner Dilemma
RandomWalking
Risk in tails
S Concurve Convex
Schellings
Shapley Value
Six Sigma
Spatial
Tipping Point
Uncertainty
Voter
EACH
3.7. 多纬度世界历史模型¶
3.8. 代理人外汇模拟(ABM Based Forex Trading Simulation)¶
第四章 数据处理平台¶
某些时候当我们的数据量足够大时（前提是有很多服务器资源），单一的计算程序已经不能够满足数据处理的需求，所以我们引入了专门的数据处理平台以优化工作。
其实，对于一般用户而言，这些平台带来的价值是小于所投入的成本的。比如某些互联网公司的数据处理平台是在其多年磨合下才达到几分钟排序100TB的效率的，而开源的直接拿来用若想达到这个效率无异于痴人说梦。
OK，扯远了，那么对目前笔者数据量不到1TB的人来说，我就看看这些平台怎么工作的，流程有没有可取的地方，然后动手撸一个自己用的模型。
2018-10: 经过长期实践后，Hadoop之类的数据处理平台远远不足以称之为“平台”，所以本章内容将注重工具的组合，即架构，让读者能够“自己搭”。
4.1 Hadoop与Spark简介（抛弃）¶
Hadoop与现在更流行的Storm和Spark，从初学的角度来说更有价值。因为Hadoop内容不止有MapReduce，更有SQL式的Yarn和HDFS这一专为MR开发的文件系统，所以我认为在基础学习阶段它更具代表性。而Storm和Spark，它们的优劣我现在并不清楚，只知道前者适用于处理输入连绵数据，后者适用于复杂MR过程的模型。
4.2 模块部署（单机/集群）¶
现在部署Hadoop的方式比过去更加容易，你可以使用 Cloudera Manager 或者 Puppet 去完成企业级的部署；如果你需要概念证明类的工作，可以直接使用 Hortonworks 的虚拟机镜像 或者 Cloudera的虚拟机镜像 ，或者 MapR ，在接下来的章节中我会使用rpm包进行安装，而不是按照 官方文档 去部署。
Hue：Hadoop User Experience ，即web UI
单节点部署¶
集群部署¶
第五章 外汇交易基础¶
对的，上来就给你看技术指标，绝对没错！
5.1 技术指标¶
Bill Williams Indicators¶
Acceleration/Deceleration Oscillator(AC)¶
Acceleration/Deceleration (AC) technical indicator signals the acceleration or deceleration of the current market driving force.
How to Use Acceleration/Deceleration
The indicator is fluctuating around a median 0.00 (zero) level which corresponds to a relative balance of the market driving force with the acceleration. Positive values signal a growing bullish trend, while negative values may be qualified as a bearish trend development. The indicator changes its direction before any actual trend reversals take place in the market therefore it serves as an early warning sign of probable trend direction changes.
The indicator color changes alone are very important as the current green line (irrespectively of the value) warns against long positions the same way as the red one would against selling.
To enter the market along with its driving force one needs to watch for both value and color. Two consecutive green columns above the zero level would suggest you to enter the market with a long position while at least two red columns below the zero level might be taken for a command to go short.
Fake signals prevail in timeframes smaller than 4 hours.
Acceleration/Deceleration Oscillator Formula (Calculation)
AC bar chart is the difference between the value of 5/34 of the driving force bar chart and 5-period simple moving average, taken from that bar chart.
AO = SMA(median price, 5)-SMA(median price, 34)
AC = AO-SMA(AO, 5)
Alligator¶
Alligator is an indicator designed to signal a trend absence, formation and direction. Bill Williams saw the alligator's behavior as an allegory of the market's one: the resting phase is turning into the price-hunting as the alligator awakes so that to come back to sleep after the feeding is over. The longer the alligator is sleeping the hungrier it gets and the stronger the market move will be.
How to Use Alligator Indicator
The Alligator indicator consists of 13-, 8- and 5-period smoothed moving averages shifted into the future by 8, 5 and 3 bars respectively which are colored blue, red and green representing the alligator’s jaw, teeth and lips.
The Alligator is resting when the three averages are twisted together progressing in a narrow range. The more distant the averages become, the sooner the price move will happen.
The averages continuing in an upward direction (green followed by red and blue) suggest an emerging uptrend which we interpret as a signal to buy.
The averages following each other in the reversed order down the slope are a strong signal of an unfolding downtrend so selling at this point would be more than appropriate.
Alligator Indicator Formula (Calculation)
MEDIAN PRICE = (HIGH + LOW) / 2
ALLIGATORS JAW = SMMA (MEDEAN PRICE, 13, 8)
ALLIGATORS TEETH = SMMA (MEDEAN PRICE, 8, 5)
ALLIGATORS LIPS = SMMA (MEDEAN PRICE, 5, 3)
Awesome Oscillator(AO)¶
Awesome Oscillator (AO) is a momentum indicator reflecting the precise changes in the market driving force which helps to identify the trend’s strength up to the points of formation and reversal.
How to Use Awesome Oscillator
There are three main signals which may be seen:
- Saucer - three consecutive columns above the nought line the first two of which must be colored red (the second one is lower than the first one) while the third one is colored green and higher than the previous (second) one. Such a formation would be a clear Buy signal whilst inverted and vertically flipped formation would serve as a Sell signal.
- Nought line crossing - the histogram crosses the naught line in an upward direction changing its values from negative to that of positive ones. In this situation we have a Buy signal. The Sell signal would be a reversed pattern.
- Two pikes - the indicator displays a Buy signal when the figure is formed by two consecutive pikes both of which are below the naught line and the later-formed pike is closer to the zero level than the earlier-formed one. The Sell signal would be given by the reverse formation.
Awesome Oscillator Trading Strategy
Awesome Oscillator Strategy includes 3 ways of trading. The first way is to open a sell position when the oscillator is below the zero line forming a peak, and open a buy position when the oscillator is above the zero line forming a gap.
Another way is to open a sell position when the oscillator forms two peaks above the zero line, where the second high is lower than the previous one. And, conversely, traders watch to open a buy position when the oscillator forms to lows below the zero line with the last one not as low as the previous one.
The third way is to account crossing the zero line. When the oscillator crosses it from up to down, it is time to open a sell position and when it crosses from down to up, it is time to open a buy position.
Awesome Oscillator Formula (Calculation)
Awesome Oscillator is a 34-period simple moving average, plotted through the central points of the bars (H+L)/2, and subtracted from the 5-period simple moving average, graphed across the central points of the bars (H+L)/2.
MEDIAN PRICE = (HIGH+LOW)/2
AO = SMA(MEDIAN PRICE, 5)-SMA(MEDIAN PRICE, 34)
where
SMA — Simple Moving Average.
Fractals¶
Fractals is an indicator highlighting the chart’s local heights and lows where the price movement had stopped and reversed. These reversal points are called respectively Highs and Lows.
How to Use Fractal Indicator
Bill Williams' Fractals are formed around a group of five consecutive bars the first two of which are successively reaching higher (or diving deeper) and the last two descending lower (or growing higher) with the middle one being the highest (or the lowest) result in the group accordingly.
Buy fractal is an arrow pointing to the top
Sell fractal is an arrow pointing to the bottom
Gator Oscillator(GO)¶
The Gator Oscillator (GO) is a supplement to the Alligator indicator and is used alongside with it showing the absolute degree of convergence/divergence of the Alligator's three SMAs pointing at the Alligator's periods of slumber and awakeness (i.e. trending and non-trending market phases).
How to Use Gator Oscillator
Being an oscillator in the form of two histograms built on either side of the naught line, the Gator Oscillator plots the absolute difference between the Alligator’s Jaw and Teeth (blue and red lines) in the positive area and the absolute difference between the Alligator’s Teeth and Lips (red and green lines) in the negative area. The histogram’s bars are colored green if exceeding the previous bar’s volume or red if falling short.
The bars of the extreme values are in tune with the strong trend forces.
The Alligator's activity periods are divided into the following four:
- Gator awakes - the bars on different sides of the naught line are colored differently.
- Gator eats - green bars on both sides of the naught line.
- Gator fills out - single red bar during the "eating" phase.
- Gator sleeps - the bars on both sides are red
.jpg)
Market Facilitation Index¶
The Market Facilitation Index is designed for evaluation the willingness of the market to move the price. The indicator's absolute values alone cannot provide any trading signals unlike it's dynamics in relation to the dynamics of the volume.
How to Use Market Facilitation Index
The absolute values of the index are represented by the histogram's bars while the comparison of the index and volume dynamics are given in colors which are vital in terms of reading the indicator signs.
Green bar - both MFI and volume are up. Increasing trading activity means market movement acceleration. We may join the trend.
Blue bar - MFI indicator is up, volume is down. The movement is continuing although the volume has dropped. The trend will soon be reversing.
Pink bar - MFI indicator is down, volume is up. The slowing down movement while volume is raising may indicate a possible break through, often a U-turn.
Brown bar - both MFI and volume are down. The market is no longer interested in the current direction and is looking for signs of a future development.
Market Facilitation Index Formula (Calculation)
BW MFI = (HIGH-LOW)/VOLUME
Oscillator¶
Aroon¶
Developed by Tushar Chande in 1995, Aroon is an indicator system that determines whether a stock is trending or not and how strong the trend is. “Aroon” means “Dawn's Early Light” in Sanskrit. Chande chose this name because the indicators are designed to reveal the beginning of a new trend. The Aroon indicators measure the number of periods since price recorded an x-day high or low. There are two separate indicators: Aroon-Up and Aroon-Down. A 25-day Aroon-Up measures the number of days since a 25-day high. A 25-day Aroon-Down measures the number of days since a 25-day low. In this sense, the Aroon indicators are quite different from typical momentum oscillators, which focus on price relative to time. Aroon is unique because it focuses on time relative to price. Chartists can use the Aroon indicators to spot emerging trends, identify consolidations, define correction periods and anticipate reversals.
Calculation
The Aroon indicators are shown in percentage terms and fluctuate between 0 and 100. Aroon-Up is based on price highs, while Aroon-Down is based on price lows. These two indicators are plotted side-by-side for easy comparison. The default parameter setting in SharpCharts is 25 and the example below is based on 25 days.
Aroon-Up = ((25 - Days Since 25-day High)/25) x 100
Aroon-Down = ((25 - Days Since 25-day Low)/25) x 100
Aroon declines as the elapsed time between a new high or low increases. 50 is the cut off point. Because 12.5 days marks the exact middle, a reading of exactly 50 is impossible on a daily chart. It is possible with other timeframes. On daily charts, Aroon is either below 50 (48) or above 50 (52). A reading above 50 means a new high or low was recorded within the last 12 days or less. This is the most recent half of the look-back period. A reading below 50 means a new high or low was recorded within the last 13 days or more {(25-13)/25 x 100 = 48). This is the latter half of the look-back period. The table below shows the range of values for 25-day Aroon-Up and 25-day Aroon-Down
Interpretation
The Aroon indicators fluctuate above/below a centerline (50) and are bound between 0 and 100. These three levels are important for interpretation. At its most basic, the bulls have the edge when Aroon-Up is above 50 and Aroon-Down is below 50. This indicates a greater propensity for new x-day highs than lows. The converse is true for a downtrend. The bears have the edge when Aroon-Up is below 50 and Aroon-Down is above 50.
A surge to 100 indicates that a trend may be emerging. This can be confirmed with a decline in the other Aroon indicator. For example, a move to 100 in Aroon-Up combined with a decline below 30 in Aroon-Down shows upside strength. Consistently high readings mean prices are regularly hitting new highs or new lows for the specified period. Prices are moving consistently higher when Aroon-Up remains in the 70-100 range for an extended period. Conversely, consistently low readings indicate that prices are seldom hitting new highs or lows. Prices are NOT moving lower when Aroon-Down remains in the 0-30 range for an extended period. This does not mean prices are moving higher though. For that we need to check Aroon-Up.
New Trend Emerging
There are three stages to an emerging trend signal. First, the Aroon lines will cross. Second, the Aroon lines will cross above/below 50. Third, one of the Aroon lines will reach 100. For example, the first stage of an uptrend signal is when Aroon-Up moves above Aroon-Down. This shows new highs becoming more recent than new lows. Keep in mind that Aroon measures the time elapsed, not the price. The second stage is when Aroon-Up moves above 50 and Aroon-Down moves below 50. The third stage is when Aroon-Up reaches 100 and Aroon-Down remains at relatively low levels. The first and second stages do not always occur in that order. Sometimes Aroon-Up will break above 50 and then above Aroon-Down. Reverse engineering the uptrend stages will give you the emerging downtrend signal. Aroon-Down breaks above Aroon-Up, breaks above 50 and reaches 100.
The chart above shows CSX Corp (CSX) with weekly bars and 25-week Aroon. First, notice that the downtrend began weakening as Aroon-Down declined below 50 at the end of 2007 (far left). The first stage of an uptrend was signaled when Aroon-Up moved above Aroon-Down in early 2008 (first orange circle). Aroon-Up continued above 50 and hit 100 as Aroon-Down remained at relatively low levels. Notice how Aroon-Up remained near 100 as the advance continued. This emerging uptrend signal lasted until September 2008 when Aroon-Down broke above Aroon-Up, exceeded 50 and surged to 100 (second orange circle). Notice how Aroon-Down remained near 100 as the downtrend extended. The third trend on this chart was signaled when Aroon-Up surged to 100 in June 2009 and remained above 50 for over a year (third orange circle). Also notice that Aroon-Down remained below 50 for over a year.
Consolidation Period
The Aroon indicators signal a consolidation when both are below 50 and/or both are moving lower with parallel lines. It makes sense that consistent readings below 50 are indicative of flat trading. For 25-day Aroon, readings below 50 mean a 25-day high or low has not been recorded in 13 or more days. Prices are clearly flat when not recording new highs or new lows. Similarly, a consolidation is usually forming when both Aroon-Up and Aroon-Down move lower in parallel fashion and the distance between the two lines is quite small. This narrow parallel decline indicates that some sort of trading range is forming. The first Aroon indicator to break above 50 and hit 100 will trigger the next signal.
The chart above shows Omnicom (OMC) with the Aroon indicators moving below 50 in a parallel decline. The width of the channel could be narrower, but we can see the consolidation taking shape on the price chart for confirmation. Both Aroon-Up and Aroon-Down were below 50 in the yellow area. Aroon-Up then broke out and surged to 100, which was before the breakout. Further confirmation came with another Aroon-Up surge at the breakout point. This surge/breakout signaled the end of the consolidation and the beginning of the advance.
The next chart shows Lifepoint Hospitals (LPNT) with 25-day Aroon. Both lines moved lower in May with a parallel decline. The distance between the lines was around 25 points throughout the decline. Aroon-Up and Aroon-Down flattened in June and both remained below 50 for around two weeks as the triangle consolidation extended. Aroon-Down (red) was the first to make its move with a break above 50 just before the triangle break on the price chart. Aroon-Down hit 100 as prices broke triangle support to signal a continuation lower.
Conclusions
Aroon-Up and Aroon-Down are complementary indicators that measure the elapsed time between new x-day highs and lows, respectively. They are shown together so chartists can easily identify the stronger of the two and determine the trend bias. A surge in Aroon-Up combined with a decline in Aroon-Down signals the emergence of an uptrend. Conversely, a surge in Aroon-Down combined with a decline in Aroon-Up signals the start of a downtrend. A consolidation is present when both move lower in parallel fashion or when both remain at low levels (below 30). Chartists can use the Aroon indicators to determine if a security is trending or trading flat and then use other indicators to generate appropriate signals. For example, chartists might use a momentum oscillator to identify oversold levels when 25-week Aroon indicates that the long-term trend is up.
RSI Bar¶
RSI-Bars is an oscillator, developed by IFC Markets in 2014 as the modification of Relative Strength Index (RSI). RSI-Bars characterizes a stability of a price momentum and allows a definition of a trend potential. A distinctive feature of RSI-Bars is that this indicator takes into account the volatility of a considered instrument within the selected timeframe - values of RSI-Bars are defined with account of price OPEN/HIGH/LOW/CLOSE (OHLC) values and are displayed in the form of chart bars. This allows avoiding of false breakdowns of oscillator trend lines and that’s why traders may use methods of a chart analysis more efficiently in this case.
Download RSI-Bars for Metatrader 4
Installation guide:
Download and extract the zip archive with indicator file .ex4;
Open the data directory from the main menu of Metatrader 4 terminal:File =>Open Data Folder;
Put an indicator file into the folder MQL4/Indicators of Data Folder;
Restart the Metatrader 4 terminal;
In order to insert an indicator, open the group of custom indicators in the main menu: Insert=>Indicators=>Custom indicator.
Advantages of RSI-Bars oscillator
In contrast to the classical Relative Strength Index, developed by J.Wilder, RSI-Bars evaluates an internal volatility. Minimal and maximum limits of bars are constructed on the basis of 4 prices (OHLC). A calculated set is used for the selection of a minimum and maximum value of RSI-Bars. Then a bar structure is formed.
An analysis of a candlestick price chart in some cases allows avoiding of a trend false breakdowns. It happens due to the account of additional price information and it internal volatility. At the same way RSI-Bars takes into account a true range of price oscillations, not only a characteristic value of a given timeframe. Due to this property, RSI-Bars allows a correct and convenient use of a chart technical analysis.
A comparative analysis of RSI and RSI-Bars is represented on the figure below – we used H4 candlesticks of a most volatile pair, GBP/USD. As it can be seen, RSI(14) has shown and additional breakdown in contrast with RSI-Bars (14). Moreover, RSI-Bars has detected later and therefore more correct finishing of a downtrend.
The use of RSI-Bars is demonstrated in trade examples of everyday analytics releases of IFC Markets.
Application
The oscillator works most efficiently in a flat motion. A lower and higher bounds of oscillator values are introduced subjectively (for example 30% and 70%) and correspond to overbought and oversold levels;
- RSI-Bars can take extreme values during a trend motion. That’s why in this case a use of overbought and oversold levels is incorrect;
- RSI-Bars allows a definition of standard chart analysis instruments - figures, lines of support and resistance, etc. In this case the indicator should be used for a confirmation of technical analysis. We should take into account that RSI-Bars can give preliminary signals of a trend change;
- Divergence is the strongest signal of RSI-Bars – opposite directions of price and oscillator movements are detected in this case. This signal is a harbinger of a possible trend weakening;
- Values of RSI-Bars lie between 0% and 100%.
Average True Range(ATR)¶
The Average True Range (ATR) indicator was introduced by Welles Wilder as a tool to measure the market volatility and volatility alone leaving aside attempts to indicate the direction. Unlike the True Range, the ATR also includes volatility of gaps and limit moves. ATR indicator is good at valuating the market's interest in the price moves for strong moves and break-outs are normally accompanied by large ranges.
How to Use ATR Indicator
The ATR is used with 14 periods with daily and longer timeframes and reflects the volatility values that are in relation to the trading instrument's price. Low ATR values would normally correspond to a range trading while high values may indicate a trend breakout or breakdown. Average True Range Indicator
Average True Range Formula (ATR Calculation)
Average True Range is a moving average of the True Range which is the greatest of the following three values:
- The distance from today's high to today's low.
- The distance from yesterday's close to today's high.
- The distance from yesterday's close to today's low.
Bollinger Bands¶
The Bollinger Bands indicator (named after its inventor) displays the current market volatility changes, confirms the direction, warns of a possible continuation or break-out of the trend, periods of consolidation, increasing volatility for break-outs as well as pinpoints local highs and lows.
How to Use Bollinger Bands
The indicator consists of the three moving averages:
- Upper band - 20-day simple moving average (SMA) plus double standard price deviation.
- Middle band - 20-day SMA.
- Lower band - 20-day SMA minus double standard price deviation.
The increasing distance between the upper and the lower bands while volatility is growing, suggests of a price developing in a trend which direction correlates with the direction of the Middle line. In contrast to the above, at times of decreasing volatility when the bands are closing in, we should be expecting the price to move sidewards in a range.
The price moving outside the Bands may indicate either the trend’s continuation (when the bands are floating apart as the volatility increases) or the U-turn of the trend if the initial movement is exhausted. Either way each of the scenarios must be confirmed by other indicators such as RSI, ADX or MACD. Anyhow the price crossing of the Middle line from below or above may be interpreted as a signal to buy or to sell respectively.
Bollinger Bands Trading Strategy
Bollinger Bands trading strategy aims to profit from oversold or overbought conditions on the market. Prices are considered overextended on the upside when they touch the upper band (overbought). They are overextended on the downside, when they touch the lower band (oversold). This strategy is used as an immediate signal to buy or sell the security. The usage of upper and lower bands as price targets is referred to as the simplest way of using Bollinger Bands strategy. If prices cross below the average, the lower band becomes the lower price target. If the prices cross above the same average, the upper band identifies the upper price target.
In a Bollinger Band trading system an uptrend is shown by prices fluctuating between upper and middle bands. In such cases if prices cross below the middle band, this warns of a trend reversal to the downside indicating a sell signal.
In a downtrend, prices fluctuate between middle and lower bands, and the price crossing above the middle band warns of a trend reversal to the upside, indicating a buy signal.
Bollinger Bands Formula (Calculation)
The middle line (ML) is a regular Moving Average:
ML = SUM [CLOSE, N]/N
The top line (TL) is ML a deviation (D) higher:
TL = ML + (D*StdDev)
The bottom line (BL) is ML a deviation (D) lower.
BL = ML — (D*StdDev)
Where:
N — number of periods used in calculation;
SMA — Simple Moving Average;
StdDev — Standard Deviation.
Commodity Channel Index(CCI)¶
The Commodity Channel Index is an indicator by Donald Lambert. Despite the original purpose to identify new trends, it’s nowadays widely used to measure the current price levels in relation to the average one.
How to Use CCI Indicator
Commodity Channel Index indicator oscillates around the naught line tending to stay within the range from -100 to +100. The naught line represents the level of an average balanced price. The higher the indicator surges above the naught line the more overvalued the security is. The further the CCI indicator plunges into the negative area the more potential for growth the price may have.
Still the unbalanced price alone may not serve as a clear indicator neither to the direction the price is following nor to its strength. There are critical values and the crossing directions which need to be looked at closely:
- Exceeding past the 100 level suggests a possible further upward movement
- Decreasing past the 100 level indicates a U-turn and serves as a signal to sell.
- Decreasing past the -100 level suggests a possible further downward movement
- Exceeding past the -100 level indicates a U-turn and serves as a signal to buy.
- Crossing the naught line upwards from below serves as a confirmation to buy
- Crossing the naught line downwards from above serves a confirmation to sell.
Smaller CCI indicator period increases its sensitivity. Shifting critical levels to 200 allows to exclude insignificant price fluctuations.
CCI Trading Strategy
CCI trading strategy is used by most traders, investors and chartists as an overbought or oversold oscillator. The basic strategy of CCI is to watch the readings above +100 and below -100. The readings above +100 are considered overbought and generate buy signals. The readings below -100 are considered oversold and generate sell signals. Though the Commodity Channel Index was initially developed for commodities, it is also used for trading stock index futures and options.
Caculation
The example below is based on a 20-period Commodity Channel Index (CCI) calculation. The number of CCI periods is also used for the calculations of the simple moving average and Mean Deviation.
CCI = (Typical Price - 20-period SMA of TP) / (.015 x Mean Deviation)
Typical Price (TP) = (High + Low + Close)/3
Constant = .015
There are four steps to calculating the Mean Deviation. First, subtract the most recent 20-period average of the typical price from each period's typical price. Second, take the absolute values of these numbers. Third, sum the absolute values. Fourth, divide by the total number of periods (20).
Lambert set the constant at .015 to ensure that approximately 70 to 80 percent of CCI values would fall between -100 and +100. This percentage also depends on the look-back period. A shorter CCI (10 periods) will be more volatile with a smaller percentage of values between +100 and -100. Conversely, a longer CCI (40 periods) will have a higher percentage of values between +100 and -100.
DeMarker(DeM)¶
This indicator was introduced by Tom DeMark as a tool to identify emerging buying and selling opportunities. It demonstrates the price depletion phases which usually correspond with the price highs and bottoms.
The DeMarker indicator proved to be efficient at identifying trend break-downs as well as spotting intra-day entry and exit points.
How to Use DeMarker Indicator
The indicator fluctuates with a range between 0 to 1 and is indicative of lower volatility and a possible price drop when reading 0.7 and higher, and signals a possible price increase when reading below 0.3.
DeMarker Indicator Formula (Calculation)
The DeMarker indicator is the sum of all price increment values recorded during the "i" period divided by the price minima:
The DeMax(i) is calculated:
If high(i) > high(i-1) , then DeMax(i) = high(i)-high(i-1), otherwise DeMax(i) = 0
The DeMin(i) is calculated:
If low(i) < low(i-1), then DeMin(i) = low(i-1)-low(i), otherwise DeMin(i) = 0
The value of the DeMarker is calculated as:
DMark(i) = SMA(DeMax, N)/(SMA(DeMax, N)+SMA(DeMin, N))
Envelopes¶
The Envelopes indicator reflects the price overbought and oversold conditions helping to identify the entry or exit points as well as possible trend break-downs.
How to Use Envelopes Indicator
The Envelopes indicator consists of two SMAs that together form a flexible channel in which the price evolves. The averages are plotted around a Moving Average in a constant percentage distance which may be adjusted according to the current market volatility. Each line serves as a margin of the price fluctuation range.
In a trending market take only oversold signals in an uptrend conditions and overbought signals in a downtrend conditions.
In a ranging market the price reaching the top line serves as a sell signal, while the price at the lower line generates a signal to buy.
Envelopes Indicator Formula (Calculation)
Upper Band = SMA(CLOSE, N)*[1+K/1000]
Lower Band = SMA(CLOSE, N)*[1-K/1000]
Where:
SMA — Simple Moving Average;
N — averaging period;
K/1000 — the value of shifting from the average (measured in basis points).
Force Index¶
The Force Index indicator invented by Alexander Elder measures the power behind every price move based on their three essential elements, e.g., direction, extent and volume. The oscillator fluctuates around the zero, i.e., a point of a relative balance between power shifts.
How to Use Force Index
The Force Index allows to identify the reinforcement of different time scale trends:
- The indicator should be made more sensitive by decreasing its period for short trends.
- The indicator should be smoothed by increasing its period for longer trends.
The Force Index may strongly imply a trend change:
- Break-down of an uptrend when the indicator's value is changing from positive to negative and price and indicator show divergence.
- Break-down of a downtrend when the indicator's value is changing from negative to positive and price and indicator show convergence.
Together with a trend-following indicator the Force Index can help identify trend corrections:
- An uptrend correction when the indicator bounces off the low.
- A downtrend correction when the indicator slides from a pike.
Force Index Formula (Calculation)
Force Index(1) = {Close (current period) - Close (prior period)} x Volume
Force Index(13) = 13-period EMA of Force Index(1)
Ichimoku¶
The Ichimoku Kinko Hyo (Equilibrium chart at a glance) is a comprehensive technical analysis tool introduced in 1968 by Tokyo columnist Goichi Hosoda. The concept of the system was to provide an immediate vision of trend sentiment, momentum and strength at a glance perceiving all the Ichimoku's five components and a price in terms of interactions among them of a cyclical type related to that of human group dynamics.
How to Use Ichimoku Indicator
The Ichimoku indicator consists of five lines which may all serve as flexible support or resistance lines, whose crossovers may as well be assumed as additional signals:
- Tenkan-Sen (Conversion line, blue)
- Kijun-Sen (Base line, red)
- Senkou Span A (Leading span A, green boundary of the cloud)
- Senkou Span B (Leading span B, red boundary of the cloud)
- Chikou Span (Lagging span, green)
Kumo (Cloud) is a central element of the Ichimoku system and represents support or resistance areas. It is formed by Leading Span A and Leading Span B.
Determining the trend persistence and corrections:
- Price moving above the cloud indicates an uptrend
- Price moving below the cloud indicates a downtrend
- Price moving within the cloud indicates a sideways trend
- Cloud turning from green to red indicates a correction during an uptrend
- Cloud turning from red to green indicates a correction during a downtrend
Determining support and resistance:
- Leading span A serves as a first support line for an uptrend
- Leading span B serves as a second support line for an uptrend
- Leading span A serves as a first resistance line for a downtrend
- Leading span B serves as a second resistance line for a downtrend
Strong Buy/Sell signals occurring above the cloud:
- Conversion line crosses Base line up from below is a signal to buy
- Conversion line crosses Base line down from above is a signal to sell
Less reliable Buy/Sell signals occurring within the cloud:
- Conversion line crosses Base line up from below is a signal to buy
- Conversion line crosses Base line down from above is a signal to sell
Ichimoku Trading Strategy
Traders use the Ichimoku strategy to identify the trend. For a bullish signal this trading strategy sets three criteria. First, the trend is bullish when prices reach above the lowest line of the cloud. Second, a bullish signal triggers when prices reverse and reach above the Conversion Line. And third, the trend is bullish when the price moves below the Base Line.
Ichimoku Formula (Ichimoku Kinko Hyo Calculation)
Tenkan-Sen (Conversion line, blue) is
(9-period high + 9-period low)/2
Kijun-Sen (Base line, red) is
(26-period high + 26-period low)/2
Senkou Span A (Leading span A, green boundary of the cloud) is
(Conversion Line + Base Line)/2
Senkou Span B (Leading span B, red boundary of the cloud) is
(52-period high + 52-period low)/2
Chikou Span (Lagging span, green) is
close price plotted 26 periods in the past
MACD¶
Moving-Average Convergence/Divergence Oscillator, commonly referred to as MACD indicator, is developed by Gerald Appel which is designed to reveal changes in the direction and strength of the trend by combining signals from three time series of moving average curves.
How to Use MACD Indicator
Three main signals generated by the MACD indicator (blue line) are crossovers with the signal line (red line), with the x-axis and divergence patterns.
Crossovers with the signal line:
- If the MACD line is rising faster than the Signal line and crosses it from below, the signal is interpreted as bullish and suggests acceleration of price growth;
- If the MACD line is falling faster than the Signal line and crosses it from above, the signal is interpreted as bearish and suggests extension of price losses;
Crossovers with the x-axis:
- A bullish signal appears if the MACD line climbs above zero;
- A bearish signal presents if the MACD line falls below zero.
Convergence/Divergence:
- If the MACD line is trending in the same direction as the price, the pattern is known as convergence, which confirms the price move;
- If they move in opposite directions, the pattern is divergence. For example, if the price reaches a new high, but the indicator does not, this may be a sign of further weakness.
MACD Indicator Formula (MACD Calculation)
MACD line = 12-period EMA – 26-period EMA
Signal line = 9-period EMA
Histogram = MACD line – Signal line
Momentum¶
Momentum Oscillator is an indicator that shows trend direction and measures how quickly the price is changing by comparing current and past prices.
How to Use Momentum Indicator
The indicator is represented by a line, which oscillates around 100. Being an oscillator, momentum should be used within price trend analysis.
Crossing the x-axis:
- It is believed that if the indicator climbs above 100 during an uptrend, it is a bullish signal;
- Otherwise if the indicator falls below 100 during a downtrend, a bearish signal appears.
Falling out of its normal range:
- Extreme points mean that the price has posted its strongest gain or loss for a particular number of moving periods, supporting trend strength;
- At the same time if the price movement was too rapid, they may indicate possible overbought and oversold areas.
Divergence patterns:
- If the price hits a new high, but the indicator does not, that could mean that investor sentiment is actually lower;
- And on the contrary if the price falls to a new low, but the indicator does not support the drop, it is a signal that the trend may end soon.
Momentum Indicator Formula (Calculation)
Momentum = (Current close price / Lagged close price) x 100
Relative Vigor Index(RVI)¶
Relative Vigor Index, developed by John Ehlers, is a technical indicator designed to determine price trend direction. The underlying logic is based on the assumption that close prices tend to be higher than open prices in a bullish environment and lower in a bearish environment.
How to Use RVI Indicator
The Relative Vigor Index allows to identify the reinforcement of price changes (and therefore may be used within convergence/divergence patterns analysis):
- Generally the higher the indicator climbs, the stronger is the current relative price increase;
- Generally the lower the indicator falls, the stronger is the current relative price drop.
Together with its signal line (Red), a 4-period moving average of RVI, the indicator (Green) may help to identify changes in prevailing price developments:
- Crossing the signal line from above, the RVI signals a possible sell opportunity;
- Crossing the signal line from below, the RVI signals a possible buy opportunity.
Relative Vigor Index Formula (RVI Calculation)
Relative Vigor Index (1) = (Close - Open) / (High - Low)
Relative Vigor Index (10) = 10-period SMA of Relative Vigor Index (1)
Relative Strenth Index(RSI)¶
Relative Strength Index is an indicator developed by Welles Wilder to assess the strength or the weakness of the current price movements and to measure the velocity of price changes by comparing price increases with its losses over a certain period.
How to Use RSI Indicator
The Relative Strength Index allows to identify possible overbought and oversold areas, but should be considered within trend analysis:
- Generally if the RSI indicator climbs above 70, the asset may be overbought;
- If the RSI indicator drops below 30, the asset may be oversold.
Leaving extreme areas the indicator may suggest possible corrections or even trend changes:
- Crossing the overbought boundary from above, the RSI signals a possible sell opportunity;
- Crossing the oversold boundary from below, the RSI signals a possible buy opportunity.
Convergence/divergence patterns may indicate possible trend weakness:
- If the price climbs to a new high, but the indicator does not, that may be a sign of the uptrend weakness;
- If the price falls to a new low, but the indicator does not, that may be a sign of the downtrend weakness.
RSI Trading Strategy
RSI trading strategy aims to generate buy and sell signals by the horizontal lines that appear on the chart at the 70 and 30 values. As we have already mentioned above, a move under 30 indicates an oversold condition and a move above 70 signals an overbought condition.
Thus, if a trader is looking for a buying opportunity, he watches the indicator dip under 30. A crossing back above 30 is considered by many traders as a confirmation that the trend has turned up. Conversely, if a trader seeks for a selling opportunity, he watches the indicator cross above the 70 line.
Relative Strength Index Formula (RSI Calculation)
RSI = 100 – 100/(1 + RS)
RS (14) = Σ(Upward movements)/Σ(|Downward movements|)
Stochastic¶
Stochastic indicator is introduced by George Lane to identify price trend direction and possible reversal points by determining the place of the current close price in the most recent price range, as in a sustainable uptrend close prices tend to the higher end of the range and to the lower end in a downtrend.
How to Use Stochastic Oscillator
The Stochastic oscillator allows to identify possible overbought and oversold areas, but should be considered within trend analysis:
- Generally if the indicator climbs above 75, the asset may be overbought;
- If the indicator drops below 25, the asset may be oversold.
Leaving extreme areas the indicator may suggest possible turning points:
- Crossing the overbought boundary from above, the Stochastic signals a possible sell opportunity;
- Crossing the oversold boundary from below, the Stochastic signals a possible buy opportunity.
Crossovers of the indicator with its smoothened signal line, usually a 3-period moving average, may also detect deal opportunities:
- The indicator suggests going long when crossing the signal line from below;
- The indicator suggests going short when crossing the signal line from above.
Convergence/divergence patterns may indicate possible trend weakness:
- If the price climbs to a new high, but the indicator does not, that may be a sign of the uptrend weakness;
- If the price falls to a new low, but the indicator does not, that may be a sign of the downtrend weakness.
Stochastic Oscillator Trading Strategy
Stochastic system is based on the observation that in an uptrend closing prices tend to be near the upper end of the price range, and in a downtrend the closing prices tend to be near the lower end of the price range.
In the Stochastic strategy two lines - the %K line and the %D line – are used. The K line is faster and the D line is slower. These lines oscillate from 0 to 100 on the vertical scale. The major signal to consider is the divergence between the D line and the price of the underlying market. When the D line is over 80 and forms two declining peaks with prices moving higher, a bearish divergence occurs. When the D line is below 20 and forms two rising bottoms with prices moving lower, a bullish divergence takes place. Thus, the actual buy and sell signals are triggered when the K line crosses the D line. A sell signal is generated when the K line crosses below the D line from above the 80 level. Accordingly, a buy signal is generated when the K line crosses above the D line bellow the 20 level.
Stochastic Oscillator Formula (Calculation)
Stochastic = 100 x ((C – L)/(H – L));
Signal = average of the last three Stochastic values;
where:
C – latest close price;
L – the lowest price over a given period;
H – the highest price over a given period.
Williams Percent Range(WPR,%R)¶
Williams Percent Range (%R) is a technical indicator developed by Larry Williams to identify whether an asset is overbought or oversold and therefore to determine possible turning points. Unlike the Stochastic oscillator Williams Percent Range is a single line fluctuating on a reverse scale.
How to Use %R
The main goal of Williams Percent Range is to identify possible overbought and oversold areas, however the indicator should be considered within trend analysis:
- Generally if the indicator climbs above -20, the asset may be overbought;
- If the indicator drops below -80, the asset may be oversold.
Leaving extreme areas the indicator may suggest possible turning points:
- Crossing the overbought boundary from above, Williams Percent Range signals a possible sell opportunity;
- Crossing the oversold boundary from below, Williams Percent Range signals a possible buy opportunity.
Divergence patterns are rare, but may indicate possible trend weakness:
- If the price climbs to a new high, but the indicator does not, that may be a sign of the uptrend weakness;
- If the price falls to a new low, but the indicator does not, that may be a sign of the downtrend weakness.
Williams %R Trading Strategy
Williams %r indicator, as already mentioned, helps to determine the points when the market is oversold or overbought. The trading rules of %R strategy are simple: buying when the market is oversold (%R reaches -80% or lower) and selling when the market is overbought (%R reaches -20% or higher).
Williams %R Formula (Calculation)
R% = - ((H - C)/(H – L)) x 100;
where:
C – latest close price;
L – the lowest price over a given period;
H – the highest price over a given period.
Trend Indicators¶
Average Directional Index(ADI)¶
Average Directional Index (ADX) is a technical indicator developed by Welles Wilder to estimate trend strength and determine probable further price movements by comparing the difference between two consecutive lows with the difference between the highs.
How to Use ADX Indicator
ADX is a complex indicator, which results from calculation of the Plus Directional Indicator (+DI – green line) and the Minus Directional Indicator (-DI – red line), but all of them may be used for trend analysis.
In general the indicator (bold line) move is believed to reflect current trend strength:
- Rising ADX (usually climbing above 25) suggests strengthening market trend – trend following indicators are becoming more useful;
- Falling ADX suggests the trend development is doubtful. ADX values below 20 may indicate neutral trend is present – oscillators are becoming more useful.
Use of complex ADX trading system may require additional confirmation signals:
- Normally if +DI (green line) climbs above -DI (red line), a buy signal is generated;
- Normally if -DI climbs above +DI, a sell signal is generated.
ADX Trading Strategy
ADX trading strategy aims to identify the strongest trends and distinguish between trending and non-trending conditions.
ADX reading above 25 indicates trend strength, while when ADX is below 25, this shows trend weakness. Breakouts, which are not difficult to spot, also help to identify whether ADX is strong enough for the price to trend or not. Thus, when ADX rises from below 25 to above 25, trend is considered strong enough to continue in the direction of the breakout.
It’s a common misperception that when ADX line starts falling this is a sign of trend reversal. Whereas, it only means that the trend strength is weakening. As long as ADX is above 25, it should be considered that a falling ADX line is simply less strong.
ADX Formula (Calculation)
ADX = MA [((+DI) – (-DI)) / ((+DI) + (-DI))] x 100;
where:
+DI – Plus Directional Indicator;
-DI – Minus Directional Indicator.
Moving Average(MA)¶
Moving Average is a technical analysis tool that shows average price over a given period of time, which is used to smoothen price fluctuations and therefore to determine trend direction and strength.
Depending of the method of averaging, distinguish between simple moving average (SMA), smoothed moving average (SMMA) and exponential moving average (EMA).
How to Use Moving Average
Generally moving average curves analysis includes the following principles:
- Direction of moving average curve reflects prevailing trend over a period;
- Low-period averaging may give more false signals, while large-period averaging tend to be lagging;
- To increase (decrease) sensitivity of the curve one should decrease (increase) the period of averaging;
- Average curves are more useful in trending environment.
Comparing moving average with price movements:
- A strong buy (sell) signal arise if price crosses from below (from above) its rising (falling) moving average curve;
- A weak buy (sell) signal arise if price crosses from below (from above) its falling (rising) moving average curve.
Comparing moving average curves of different periods:
- A rising (falling) lower-period curve crossing from below (above) another rising (falling) longer-period curve gives a strong buy (sell) signal;
- A rising (falling) lower-period curve crossing from below (above) another falling (rising) longer-period curve gives a weak buy (sell) signal.
Moving Average Trading Strategy
Moving average strategy is essentially a trend following means. Its objective is to signal the beginning of a new trend or a trend reversal. Herein, its main purpose is to track the progress of the trend and not to predict market action in the same sense that technical analysis attempts to do. By its nature, Moving Average is a follower; it follows the market telling that a new trend has begun or reversed only after the fact.
Moving Average Formula (Calculation)
SMA = Sum (Close (i), N) / N,
where:
Close (i) – current close price;
N – period of averaging.
EMA(t) = EMA(t-1) + (K x [Close(t) – EMA(t-1)]),
where:
t – current period;
K = 2 / (N + 1), N – period of averaging.
SMA¶
Generally, the term ''Moving Average'' refers to Simple Moving Average. The latter does not predict price direction; it is a lagging indicator and rather defines the current direction. It is an indicator that shows the average value of the instrument's price over a specified period of time.
Simple Moving Average Example
An SMA is calculated by adding the closing price of the instrument to the number of time periods and then dividing the total number by the number of time periods. The result will be the average price of the instrument over a certain time period. Thus, in order to calculate a 10-day SMA, it's necessary to add closing prices over a 10-day period and divide the total number by 10. As the term ''moving'' implies, prices move according to the point on the chart. This means that always a new calculation is needed that can correspond to the time period of the average used. Thus, you can recalculate a 10-day average by adding the new day and missing out the 10th day and so on.
Though simple moving average is used by most traders and analysts, it is criticized by two reasons. The first criticism is that only the time period covered by the average is taken into consideration. And secondly, the SMA gives equal weight to each day's price.
Nevertheless, Simple Moving Average has become a preferred method for tracking prices due to its simplicity and quick calculation. By the same simplicity early market analysts performed the market analysis without using complicated chart metrics that are widely applied today. They mainly relied on market prices as the main means of tracking trends and market direction. This process was boring but was confirmed to be profitable and reliable, and up till now it continues to be a popular technical analysis tool extensively used by most traders.
Moving Average of Oscillator(OsMA)¶
Moving Average of Oscillator (OsMA) is a technical analysis tool that reflects the difference between an oscillator (MACD) and its moving average (signal line).
How to Use OsMa Indicator
Extremum points:
- OsMA switching from falling to rising in extreme areas may be a sign of bullish reversal;
- OsMA switching from rising to falling may be a sign of bearish reversal.
Crossing zero axis:
- OsMA rising above zero (corresponds to MACD crossing from below its signal line) generates a buy signal;
- OsMA falling below zero (corresponds to MACD crossing from above its signal line) generates a sell signal.
Moving Average of Oscillator Formula (Calculation)
OsMA = MACD – Signal
Parabolic(SAR)¶
Parabolic is a trend following indicator developed by Welles Wilder and designed to confirm or reject trend direction, to determine trend end, correction or flat stages as well as to indicate possible exit points. The underlying principle of the indicator can be described as “stop and reverse” (SAR).
How to Use Parabolic SAR
When using the indicator we should take into consideration its positioning against the price chart as well as its acceleration factor which increases together with the trend. Despite being a popular tool of analysis, it has limitations and may give false signals in frequently changing market conditions.
The indicator may signal the following:
Trend confirmation
- If the indicator is plotted below the price graph, it stands for an uptrend;
- If the indicator is plotted above the price graph, it confirms a downtrend.
Exit points determination
- If the price drops below Parabolic line during an uptrend, there may be sense in closing long positions;
- If the price rises above Parabolic curve during a downtrend, there may be sense in closing short positions.
Signal significance is determined with the use of the acceleration factor. The acceleration factor increases each time the close price is higher than its previous value in an uptrend and lower in a downtrend. It is believed that the indicator is more reliable when the price’s and the indicator’s moves are parallel and less reliable when they converge.
Parabolic SAR Formula (Calculation)
P(t) = P(t-1) + AF x (EP(t-1) – P(t-1)),
where:
P(t) – current value of the indicator;
P(t-1) – value in the previous period;
AF – acceleration factor, generally rising from 0.02 to 0.2 with a step of 0.02;
EP(t-1) – extreme price in the previous period.
ZigZag¶
The ZigZag feature on SharpCharts is not an indicator per se, but rather a means to filter out smaller price movements. A ZigZag set at 10% would ignore all price movements less than 10%. Only price movements greater than 10% would be shown. Filtering out smaller movements gives chartists the ability to see the forest instead of just trees. It is important to remember that the ZigZag feature has no predictive power because it draws lines base on hindsight. Any predictive power will come from applications such as Elliott Wave, price pattern analysis or indicators. Chartists can also use the ZigZag with retracements feature to identify Fibonacci retracements and projections.
Calculation
The ZigZag is based on the chart “type.” Line and dot charts, which are based on the close, will show the ZigZag based on closing prices. High-Low-Close bars (HLC), Open-High-Low-Close (OHLC) bars and candlesticks, which show the period's high-low range, will show the ZigZag based on this high-low range. A ZigZag based on the high-low range is more likely to change course than a ZigZag based on the close because the high-low range will be much larger and produce bigger swings.
The parameters box allows chartists to set the sensitivity of the ZigZag feature. A ZigZag with 5 in the parameter box will filter out all movements less than 5%. A ZigZag(10) will filter out movements less than 10%. If a stock traded from a reaction low of 100 to a high of 109 (+9%), there would not be a line because the move was less than 10%. If the stock advanced from a low of 100 to a high of 110 (+10%), there would be a line from 100 to 110. If the stock continued on to 112, this line would extend to 112 (100 to 112). The ZigZag would not reverse until the stock declined 10% or more from its high. From a high of 112, a stock would have to decline 11.2 points (or to a low of 100.8) to warrant another line. The chart below shows a QQQQ line chart with a 7% ZigZag. The early June bounce was ignored because it was less than 7% (black arrow). The two pullbacks in July were ignored because they were much less than 7% (red arrows).
Be careful with the last ZigZag line. Astute chartists will notice that the last ZigZag line is up even though QQQQ advanced just 4.13% (43.36 to 45.15). This is just a temporary line because QQQQ has yet to reach the 7% change threshold. A move to 46.40 is needed for a gain of 7%, which would then warrant a permanent ZigZag line. Should QQQQ fail to reach the 7% threshold on this bounce and then decline below 43, this temporary line would disappear and the prior ZigZag line would continue from the early August high.
Elliott Wave Counts
The ZigZag feature can be used to filter out small moves and make Elliott Wave counts more straight-forward. The chart below shows the S&P 500 ETF with a 6% ZigZag to filter moves less than 6%. After a little trial and error, 6% was deemed the threshold of importance. An advance or decline greater than 6% was deemed significant enough to warrant a wave for an Elliott count. Keep in mind that this is just an example. The threshold and the wave count are subjective and dependent on individual preferences. Based on the 6% ZigZag, a complete cycle was identified from March 2009 until July 2010. A complete cycle consists of 8 waves, 5 up and 3 down.
Retracements and Projections
Sharpcharts users can choose between the normal “ZigZag” and “ZigZag (Retrace.).” As shown in the examples above, the normal ZigZag shows lines that move at least a specific percentage. The ZigZag (Retrace.) connects the reaction highs and lows with labels that measure the prior move. The numbers on the dotted lines reflect the difference between the current Zigzag line and the ZigZag line immediately before it. For example, the chart below shows Altera (ALTR) with the 15% ZigZag (Retrace.) feature. Three ZigZag lines have been labeled (1, 2 and 3). The dotted line connecting the low of Line 1 with the low of Line 2 shows a box with 0.638. This means Line 2 is .638 (63.8%) of Line 1. A number below 1 means the line is shorter than the prior line. The dotted line connecting the high of Line 2 with the high of Line 3 shows a box with 1.646. This means Line 3 is 1.646 (164.6%) of Line 2. A number above 1 means the line is longer than the prior line.
As you may have guessed, seeing these lines as a percentage of the prior lines makes it possible to assess Fibonacci retracements Fibonacci projections. The August decline (Line 2) retraced around 61.8% of the June-July advance (Line 1). This is a classic Fibonacci retracement. The advance from early September to early November was 1.646 times the August decline. In this sense, the ZigZag (Retrace.) can be used to project the length of an advance. Again, 1.646 is close to the Fibonacci 1.618, which is the Golden Ratio used in many projection estimates. See our ChartSchool article for more on Fibonacci retracements.
Conclusions
The ZigZag and ZigZag (Retrace.) filter price action and do not have any predictive power. The ZigZag lines simply react when prices move a certain percentage. Chartists can apply an array of technical analysis tools to the ZigZag. Chartists can perform basic trend analysis by comparing reaction highs and lows. Chartists can also overlay the ZigZag feature to look for price patterns that might not be as visible on a normal bar or line chart. The ZigZag has a way of highlighting the important movements and ignoring the noise. When using the ZigZag feature, don't forget to measure the last line to determine if it is temporary or permanent. The last ZigZag line is temporary if the current price change is less than the ZigZag parameter. The last line is permanent when the price change is greater than or equal to the ZigZag parameter.
Volume Indicators¶
Accumulation/Distribution(AD)¶
Accumulation/Distribution is a volume-based technical analysis indicator designed to reflect cumulative inflows and outflows of money for an asset by comparing close prices with highs and lows and weighting the relation by trading volumes.
How to Use Accumulation/Distribution
The Accumulation/Distribution line is used for trend confirmation or possible turning points identification purposes.
Trend confirmation:
- An uptrend in prices is confirmed if A/D line is rising;
- A downtrend in prices is confirmed if A/D line is falling.
Divergence pattern analysis:
- Rising A/D line along with decreasing prices indicates the downtrend may be weakening to a bullish reversal;
- Falling A/D along with rising prices indicates the uptrend may be weakening to a bearish reversal.
Accumulation/Distribution Indicator Formula (Calculation)
A/D(t) = [((C – L) – (H – C)) / (H – L)] x Vol + A/D(t-1),
where:
A/D(t) – current Accumulation/Distribution value;
A/D(t-1) – previous Accumulation/Distribution value;
H – current high;
L – current low;
C – close price;
Vol – volume.
Money Flow Index(MFI)¶
Money Flow Index (MFI) is a technical indicator developed to estimate money inflow intensity into a certain asset by comparing price increases and decreases over a given period, but also taking into consideration trading volumes. How to Use Money Flow Index
The indicator can be used to identify whether an asset is overbought or oversold, as well as to determine possible turning points.
Analyzing extreme (overbought/oversold) areas:
- If MFI climbs above 80, the asset is generally considered to be overbought. A sell signal appears if MFI crosses the overbought area boundary from above;
- If MFI drops below 20, the asset is generally considered to be oversold. A buy signal appears if MFI crosses the oversold area boundary from below.
Divergence patterns analysis:
- Rising MFI along with decreasing prices indicates the downtrend may be weakening;
- Falling MFI along with rising prices indicates the uptrend may be weakening.
Money Flow Index Formula (Calculation)
The following steps are required to calculate the index:
1. TP = (H + L + C) / 3;
2. MF = TP*Vol;
3. MR = Sum(MF+) / Sum(MF-);
4. MFI = 100 – (100 / (1 + MR)),
where:
TP – typical price;
H – current high;
L – current low;
C – close price;
MF – money flow (positive (MF+) if current TP > previous TP, negative (MF-) otherwise);
Vol – volume;
MR – money ratio.
On-Balance Volume(OBV)¶
On-Balance Volume (OBV) is a cumulative volume-based tool intended to show the relation between the amount of deals and asset’s price movements.
How to Use On Balance Volume
The On-Balance Volume line is used for trend confirmation or possible turning points identification purposes.
Trend confirmation:
- An uptrend in prices is confirmed if the line is rising;
- A downtrend in prices is confirmed if the line is falling.
Divergence pattern analysis:
- Rising OBV line along with decreasing prices indicates the downtrend may be weakening to a bullish reversal;
- Falling OBV along with rising prices indicates the uptrend may be weakening to a bearish reversal.
On-Balance Volume Formula (Calculation)
OBV(t) = OBV(t-1) + Vol, if C(t) > C(t-1);
OBV(t) = OBV(t-1) – Vol, if C(t) < C(t-1);
OBV(t) = OBV(t-1), if C(t) = C(t-1),
where:
t – current period;
t-1 – previous period;
C – close price;
Vol – volume.
Volumes¶
Volume indicator is a technical analysis tool, which reflects trading activity of investors for a given time period.
How to Use Volume Indicator
Volume indicator is generally used together with price analysis to confirm trend strength or highlight its weakness and therefore identify possible upcoming reversals.
Trend confirmation:
- Rising trading volumes during an uptrend confirms bullish mood;
- Rising trading volumes during a downtrend confirms bearish mood.
Trend weakness:
- If volumes are falling while prices are increasing, that may be a sign of uptrend weakness, as demand for the asset may cease at higher prices.
Forex Volumes Calculation
Volume = total value/number of transactions during a given period.
5.2 自动交易基础¶
笔者编写了一些交易脚本，包括EA和script，可访问https://github.com/lofyer/mt4-scripts进行下载。
5.3 基本分析基础¶
央行测量啦、社会指标啦、等等啦啦啦
第六章 信号交易系统（外汇、股市与区块链）¶
先当作普通的控制系统处理，然后加入更多输入信号之后当作复杂系统处理，采用ABM建模。
系统输入信号：
技术分析、舆论分析、新闻分析（含突发新闻与数据新闻）、报告分析，
以及它们的历史数据、机器学习后backtesting的数据。
系统控制器：
为了保持盈利水平，可能需要扩大正反馈，减弱负反馈，即止盈大、止损小。
控制器设计在输入信号作为反馈的基础上，将历史订单的盈利水平作为主要参考，
即将整个控制器置于输入环节，从而形成完整的闭环系统。
软件架构：
为了保持高并发，哈希环分发任务，消息队列mq，使用工作流控制任务，数据库使用少字段多KV，新增功能加到KeyVaule列表中，无需增加数据库字段。
系统平台流程概述：
交易：
注册 -> 分配MT4虚拟机（容器）-> 交易开始 -> 周期结算
注册 -> 独立MT4客户端 -> 交易开始 -> 周期结算
新闻：
新闻提供GIS与新闻源两种查看方法
免注册 -> 新闻查看、搜索
注册 -> 新闻查看、搜索、推送
算法：
一：专家人工复制
二：指数指标机器学习交易
三：基本分析与舆情分析，并结合（二）进行交易
外汇信号系统即是面向外汇新手的平台，普遍具备最基本的（文字）新闻发布功能，再者即是合适买入或卖出的信息，如果合适的话，也可以提供交易拷贝（trade copier）让其他人自动跟随专家的交易方式进行交易。
当前整个系统的架构比较简单，总体可看作一个非线性回馈控制系统，输入信号包括新闻、新闻对外汇的评级（以下简称评级）、技术指标、历史订单、其他网络信号，输出则简化为订单处理（下单、平仓、修改）与事件（用于推送到客户与网站），其中订单的盈利情况会作为回馈信号。
画个图可能更清晰：
这是目前的服务器架构图：
目前与本章相关的项目：
主站点
https://forex.fusionworks.cn
新闻GIS系统
https://github.com/lofyer/forex-gis-news.git
MT4服务端
https://github.com/lofyer/mt4-server.git
MT4交易脚本
https://github.com/lofyer/mt4-scripts.git
TUI汇率展示
https://github.com/lofyer/forex-rate-board.git
OANDA API调用脚本
https://github.com/lofyer/oanda-scripts.git
交易模型
https://github.com/lofyer/quantum-trade-model.git
原始交易模型
https://github.com/lofyer/trade-model-01.git
在学习分类指标与评级指标的时候，我们需要做批量的backtesting。
6.1. 新闻、报告与大盘信息¶
6.1.1. 新闻收集¶
新闻收集使用了SimplePie，对各大主流媒体的RSS进行每小时一次的收集活动，并将其存储至MySQL/Oracle数据库。
6.1.2. 新闻分类与评级¶
新闻分类我们可以参考 Google的专利 ，即选择某类新闻的频率较高的词，然后再将其应用到新的新闻中。笔者觉得此时监督学习的效率更高。
StoryId Class word1 word2 gram1 gram2 ...
1 sports 0 0.2 0.01 0
2 tech 0.5 0.01 0 0.3
3 sports 0 0.1 0.3 0.01
分词、NLTK工具、分类
https://github.com/yassersouri/classify-text
https://github.com/kareemf/news-article-classification
scipy
scikit_learn
pybrain
jieba
tensorflow
可直接使用python-geograpy进行新闻地点筛选，以及pycountry查询地点信息。
评级需要学习 ，即根据新闻发生时间、地点、事件，然后结合大盘状况进行评级。
6.1.3. 报告评级¶
报告不同于新闻，它的权威性与概括性更强，时效跨度或长或短。目前，对于计算机来说，报告的处理难以达到人工理解的水平，因为它需要结合尽可能多的信息去处理，而对于一般交易者来说，这需要相当高的技术要求。
报告评级的权重高于新闻评级。以笔者经验来看有以下报告会对汇率产生重大波动：
- 国际收支（贸易余额）
- 利率与货币供给政策
- 通胀与通缩
- 资产价格（债券、股票、房地产）
- 商品价格（CRB现货指数）
- 政府预算与财政政策
- 国家信用评级
- 政治与战争
- 自然因素
- 数据公布（消费者物价指数CPI、零售销售、生产者物价指数PPI、工业生产IP、采购经理人指数PMI、就业数据）
6.1.4. GIS展示¶
可参考网站内容
6.1.5. 大盘历史数据¶
获取历史数据的途径比较多，Python/R语言中有非常方便的库可供直接调用，比如pandas、quantmod等。但是，他们的数据源、数据周期、数据精度等不太满足我们的回测需求，所以在经过大量测试后，笔者推荐使用如下方式进行获取。
QCollector Expert for eSignal
MT4客户端数据下载
6.3. 相关性处理¶
机器学习部分将技术指标、新闻与大盘数据进行相关性学习，其处理过程可被随时查看与人工干预。
6.4. 交易策略¶
6.4.1. 策略1 - 批量¶
6.4.2. 策略2 - 止损批量¶
6.4.4. 策略4 - 技术指标与新闻自动交易¶
6.4.5. 策略5 - 基于历史数据机器学习的技术指标与新闻自动交易¶
6.4.5. 策略6 - 代理人模型与复杂性策略应用¶
6.5. 回馈信号的处理¶
6.6. 输出处理¶
客户交易端与新闻端
6.8. 交易箴言¶
网络代理
FPGA交易机器人
网络代理
Dow Jones Theory¶
交易心理¶
In the contemporary exchange market exchange rates are defined through decisions of thousands of traders and investors. The psychology of human behavior is considered to be the clue of understanding what happens in financial markets. What are the motivations for trading? How our emotions affect our decicion making process? How to avoid the failure and become a successful trader? Have you ever wondered about these questions? Keep reading to find out the answers.
Avoiding Failure
In stock trading decisive influence on the behavior of the trader is made by common to all feelings such as fear, greed, hope, etc. Weak and self confident, greedy and slow; all these people are doomed to become the victims of the market.
The recognition of your own abilities, positive or negative qualities will help you as a trader to avoid failure. If we also add to this the ability of adequate evaluation of the psychological state and the behavior of the market, the success is guaranteed.
The Motivations
One of the driving forces, making you to take part in the work of speculative financial markets, is the possibility of earning "easy money" or, saying directly, greed. The result of greedy action is the motivation for making deals.
One can distinguish between two kinds of motivations:
- Rational motivation is expressed through cold prudence when taking decisions about making a deal.
- Irrational motivation is expressed through passion of the player; the others are the slaves of their emotions and are practically doomed to lose.
- If the trader does not have a working plan formed before making deals, it speaks about the fact that the person is likely to work under the influence of greed but not reason.
Understanding Hope
The following factor motivating the trader to make deals is the hope to get profit. If the hope prevails over the profit calculation, the trader undertakes the risk of overestimating his abilities when analyzing the situation. Hope must be placed in subordinate relations both with calculation and greed. It is the great hope that brings beginners to failure. The trader, living with hope, is doomed to failure. It is a hope that pushes traders towards making one of the most cruel mistakes- shift of the stop-loss orders level.
From outside, trading seems to be an utterly simple matter. But in reality for the majority of people it later on appears to be the most difficult of all the issues.
Accepting Losses
You will not be able to become a successful trader until you are ready both for victories and losses. Both of them are important and inseparable parts of the trading process. On the way of mastering the art of trade very often barriers are met. When the trader focuses on the problems (there can be numerous problems, for instance, lack of means, resources and knowledge), he feels anger, guilt, disappointment and dissatisfaction. But such an emotional state will not let him move forward. If the loss is unacceptable for the trader, he will not be able to close the losing position. When the trader is not ready to face losses, they usually become more.
Trading Psychology and Self-Discipline
In trading, there is a tiny minority of winners and overwhelming majority of losers and the latter wish to know the secrets of success of the winners. But is there a difference between them? Yes, there is; the one who makes money week by week, month by month and year by year, trades keeping self-discipline. To the question of the secrets of his stable market triumph, such a winner answers without hesitating, that he was able to reach such heights by learning how to control his emotions and change his decisions to match the market.
Note, self discipline, control of emotions and the ability to reconsider are all psychological moments which are not related to information services, consultation firms, new exchanges, technical or fundamental trade systems (with computer programs or without them).
Do not confuse confidence with extreme self-confidence
Interviews with traders confirm that extreme self-confidence plays an important role in making trading decisions. If the trader receives good profit, he becomes more prone to risk which is followed by negative consequences. This is a tendency of becoming extremely self-confident after success, which mostly happens with the less experienced market participants.
Extreme confidence easily transforms into a dangerous quality, as people who are too much confident in their beliefs will not pay attention to important information which is valuable for their trading decisions. Confidence and negative emotions are directly related to each other in strength. In general, confidence and fear are similar senses by nature; only the one is with a "plus" sign and the other with - "minus" sign. If the person feels more confident, there is a little room left for confusion, alarm and fear.
How does the sense of self confidence develop?
In a natural way, the person gets used to relying on himself in everything that he has to do without any hesitation. With such trust in himself he does not have to fear the market with its seemingly unpredictable and chaotic behavior. The matter here is not with him at all, as the market did not change but the inner world and psychological warehouse of the trader have.
How to become a successful trader?
There are two important terms in relation to a good trader.
- To set a principle of trading exclusively on the basis of self discipline.
- To learn how to remove the negative emotional energy of the last trade experience.
Due to the principle to self discipline, self trust is being formed, which is necessary for successful trading actions.
Almost in the majority of cases each trader starts his way on the primary level without understanding trading psychology and without the principle of self discipline. And it is likely to get psychological trauma (a psychological state which is capable of making people feel fear) of this or that severity. It is necessary to learn how to get rid of worries. When there is little fear as a consequence, you absorb new knowledge about the nature of the market.
Do not forget that each moment is an excellent indicator of your development level. But if you consider each failure (if it did not happen as you have expected or wanted) to be a mistake, you very often deprive yourself of understanding yourself. While people become shy of learning something new about them. Why? Because mistakes mean an emotional pain for them. Avoiding pain instinctively, the person unconsciously refuses to recognize himself, when it is necessary to manage better a similar situation in future.
The bottom line
To reach a success in trading, you need to take the whole responsibility of your decisions and actions on yourself.
6.9 全链路测试代理¶
第七章 指导白皮书¶
1. 软硬件配置¶
算法设计完成后如果有高频需求，可能需要设计FPGA。
目前采用Xilinx的Zynq MPSoC以及Kintex-7两款FPGA，基础设施以及业务层面，可加速的内容都非常多，包括负载均衡、算法固化、加解密、网络卸载等。
HPC集群暂时使用ARM+NVIDIA架构，无实时处理部分，仅训练以及函数计算服务。
考虑到部分数据传输延迟问题，会采用CPLD/FPGA或专用无线电微波传输，频率在6GHz以上（低于此的意义不大），目前成熟产品为80Ghz，速率为3Gbps。如此以来，伦敦到法兰克福（396英里，673公里）的延迟可从传统光传输所花费的17ms降至4.2毫秒。
一直想搞台大型机，哪怕是玩玩扫雷也成，所以基础设施中先加个玩具大型机。
1.1. FPGA处理技术指标¶
1.2. 基于ARM与NVIDIA GPU或FPGA TPU的HPC集群¶
1.3. FPGA加速机器学习框架¶
1.4. 无线电微波传输降低延迟¶
1.5. IBM大型机模拟器Hercules（迁移）¶
1.6. 高性能虚拟化¶
2. 交易备注¶
- 来源函数、即时指标
TODO:
合并5、7两章。
第4章要不要了？要吧，回头简化一下，注重搜索引擎与数据降维即可。
第七章以及以后的就别看了。