Before we get started I need to clarify something. Theoretical decision trees can have two or more branches protruding from a single node. However, this can be computationally expensive so most implementations of decision trees only allow binary splits. Recall our example problem – Bill is a user on our online dating site and we want to build a decision tree that predicts whether he would message a certain woman. (If so, we say that she’s “date-worthy”).
A decision tree is a model that uses a set of criteria to classify something. Suppose you tell your single friend Bill to go out with your new friend Sally. Since Bill has never met Sally, he asks you a series of questions. Bill: How far from me does she live? You: 15 miles Bill: How tall is she? You: 5’6 Bill: Does she have a college degree? You: No