Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms
Web-scraping and data mining algorithms are used extensively by hedge funds, equities traders, digital marketers and in the technology sector more broadly. Contrastingly, the real estate development industry continues to use traditional, manual methods to identify and pursue new development opportun...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Published: |
Massachusetts Institute of Technology
2022
|
| Online Access: | https://hdl.handle.net/1721.1/139272 |
| _version_ | 1826192477278699520 |
|---|---|
| author | Williams, Oscar |
| author2 | Wheaton, William |
| author_facet | Wheaton, William Williams, Oscar |
| author_sort | Williams, Oscar |
| collection | MIT |
| description | Web-scraping and data mining algorithms are used extensively by hedge funds, equities traders, digital marketers and in the technology sector more broadly. Contrastingly, the real estate development industry continues to use traditional, manual methods to identify and pursue new development opportunities with the exception of mapping software which has been widely adopted. The lack of adoption of these technologies is primarily due to the difficulty in identifying, retrieving and processing the required data rather than an inherent lack of data. To the contrary, there is a wealth of public and private information available to the real estate development industry that can provide value if collected and analyzed efficiently and at scale using algorithms. To test this hypothesis, the author has built a functioning web-scraping and data collection platform that demonstrates how large amounts of data can be retrieved and processed at scale. This thesis evaluates the effectiveness of using web-scraping algorithms to search for real estate development and land rezoning opportunities from publicly available local Government data. The focus area of the thesis is Sydney, Australia and the subject of the thesis is the Aiden1 platform that is owned by the Principal Investigator and author. The platform uses automated web-scraping algorithms to parse publicly available local Government data for keywords that indicate a prospective development opportunity or an instance of imminent land rezoning. The results of this research demonstrate the effectiveness of adopting web-scraping technologies and the usefulness to real estate development professionals. |
| first_indexed | 2024-09-23T09:15:44Z |
| format | Thesis |
| id | mit-1721.1/139272 |
| institution | Massachusetts Institute of Technology |
| last_indexed | 2024-09-23T09:15:44Z |
| publishDate | 2022 |
| publisher | Massachusetts Institute of Technology |
| record_format | dspace |
| spelling | mit-1721.1/1392722022-01-15T04:00:53Z Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms Williams, Oscar Wheaton, William Massachusetts Institute of Technology. Center for Real Estate. Program in Real Estate Development. Web-scraping and data mining algorithms are used extensively by hedge funds, equities traders, digital marketers and in the technology sector more broadly. Contrastingly, the real estate development industry continues to use traditional, manual methods to identify and pursue new development opportunities with the exception of mapping software which has been widely adopted. The lack of adoption of these technologies is primarily due to the difficulty in identifying, retrieving and processing the required data rather than an inherent lack of data. To the contrary, there is a wealth of public and private information available to the real estate development industry that can provide value if collected and analyzed efficiently and at scale using algorithms. To test this hypothesis, the author has built a functioning web-scraping and data collection platform that demonstrates how large amounts of data can be retrieved and processed at scale. This thesis evaluates the effectiveness of using web-scraping algorithms to search for real estate development and land rezoning opportunities from publicly available local Government data. The focus area of the thesis is Sydney, Australia and the subject of the thesis is the Aiden1 platform that is owned by the Principal Investigator and author. The platform uses automated web-scraping algorithms to parse publicly available local Government data for keywords that indicate a prospective development opportunity or an instance of imminent land rezoning. The results of this research demonstrate the effectiveness of adopting web-scraping technologies and the usefulness to real estate development professionals. S.M. 2022-01-14T15:00:46Z 2022-01-14T15:00:46Z 2021-06 2021-05-26T14:41:39.920Z Thesis https://hdl.handle.net/1721.1/139272 In Copyright - Educational Use Permitted Copyright retained by author(s) https://rightsstatements.org/page/InC-EDU/1.0/ application/pdf Massachusetts Institute of Technology |
| spellingShingle | Williams, Oscar Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms |
| title | Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms |
| title_full | Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms |
| title_fullStr | Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms |
| title_full_unstemmed | Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms |
| title_short | Identifying Real Estate Development Opportunities: Web-Scraping, Regex Patterns & String-Searching Algorithms |
| title_sort | identifying real estate development opportunities web scraping regex patterns string searching algorithms |
| url | https://hdl.handle.net/1721.1/139272 |
| work_keys_str_mv | AT williamsoscar identifyingrealestatedevelopmentopportunitieswebscrapingregexpatternsstringsearchingalgorithms |