Experiencias con Golang (Patricio Pérez)

Please download to get full document.

View again

All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
 3
 
  1. Golang, a Swiss Army Knife for DevOps AKA: Experience with Go in client Patricio Pérez | p.perez.bustos@accenture.com 2. Índice ○ A Little briefing story ã What…
Share
Transcript
  • 1. Golang, a Swiss Army Knife for DevOps AKA: Experience with Go in client Patricio Pérez | p.perez.bustos@accenture.com
  • 2. Índice ○ A Little briefing story • What is Go? (wiki at it best) • Advantages (or what was meant to solve) • Expectations • Why so loved? • Who use Go, anyway? ○ Experience ○ Demo
  • 3. A little briefing story What is Golang, and how can it be compared with other languages?
  • 4. Quote for Golang (found in web) “Oh I love Go. I’ve spent more time coding than debugging and it’s so simple, fast and funny…” – Roberto Costumero (@rcostu) “Programming in Go is like being young again (but more productive!).” – Anneli
  • 5. What is Go? (wiki at it best) - A systems-level programming language (also known as Low-Level language) - Focused for large, distributed systems and highly-scalable network servers. - Was meant to alleviate some of the slowness and clumsiness of development of very large software systems. - Developed in 2007 by Robert Griesemer, Rob Pike (co-creator of UTF-8), and Ken Thompson (co-Creator of UNIX), inside Google. - 1.0 launched in 2012 - Actual version: 1.12 (February 2019)
  • 6. Advantages (or what was meant to solve) ○ Fast Compilation & Execution. ○ Compatibility with other programming languages. ○ Nice readability and documentation (if you are used to C or Java). ○ Language consistency (Grammar sensible). ○ Multi-platform builds (Binaries for any platform). ○ Can be executed or compiled (similar to PERL). ○ Best use of routines and forks (concurrency). ○ Plugins (imports) everywhere! GitHub / local import.
  • 7. Expectations https://insights.stackoverflow.com/survey/2018/#most-loved-dreaded-and-wanted
  • 8. Expectations https://insights.stackoverflow.com/survey/2018/#most-loved-dreaded-and-wanted
  • 9. https://research.hackerrank.com/developer-skills/2018/#insight4
  • 10. Why so loved?
  • 11. “Go is like a very delicious trifle - the further into it you go, the more delicious things you find. The quality is clear throughout. Go is very unassuming. You start by wondering what the big deal is + getting annoyed with the minor differences from other C-syntax languages before slowly progressing towards quite liking it, then eventually once you grok how simple + elegant and well engineered it is you come to love it.” Lorenzo Stoakes Fuente: https://news.ycombinator.com/item?id=3770624
  • 12. Who use Go, anyway?
  • 13. Who use Go, anyway?
  • 14. Experience My Experience in Client-Side
  • 15. 1) Regular Expressions Experiences with Golang in DevOps 1st Version (PMD Compare): - Script to compare names in the files, check if the project is the same and use it in a comparisson script (already implemented). - Bash (Little knowledge: 8 hrs) - Golang (Little knowledge: 2 hrs) Key: RegExp handling. 2nd Version (PMD Compare): - The same script to compare the content of 2 zips and generate a full report. - Golang: 24 hrs. Without Report: 2 hrs each rev step. With Report: 5 min. 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 16. 1) Regular Expressions Experiences with Golang in DevOps Problem: - Platform was web, without Access to DB (SaaS) [Changepoint] - Used NTLM (Windows Authentication) to Access. - Process was slow, unorthodox and boring... - Page visualization was awful. - The page had attachments. - The user was used for many members at the same time. - Limited concurrent users to 1. Solution: Golang Script: - Took 2 weeks to develop. (~520 LoC) - Get & Post Handling to emulate process. - HTML parsing & fetching. - Generated page from Template, with Bootstrap. - Centralized Access through Jenkins. - Plugins Used: - NTLM Authentication plugin - HTML handled as jQuery (GoQuery) Old Process: 30 min. Golang Script fetching: 5 sec (10 sec total) 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 17. 1) Regular Expressions Experiences with Golang in DevOps Problem: - Remote machine (Windows) sometimes freezes (Many JVM with Weblogic) - Happened on Friday (Home Office) - Used by local & outsourcing team (India, Monday was wasted). - Needed to restart without powershell privileges. - Only access through console (cmd). Solution: Golang Script: - Took 1 hour - Generated Windows binaries (32/64 bit) - Jenkins job to call the executable. Worked everytime. 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 18. 1) Regular Expressions Experiences with Golang in DevOps Problem: - Once a week, the client needed to load to Production a DB & LDAP script. - The data came from an Excel file. - The Excel sometimes came with error for special Characters. - Nodoby was in charge for LDAP Server. - DB ticket takes around a day to solve (QA Validation first). - Subject to errors. - Process take around 1 hour to half day. - Repetitive. Solution: Golang Script: - Took 1 week (and around 5 iterations, because of limitations). - Iterations only took a day each one. - Modularized for better maintenance. - User can execute script from Jenkins. - Execution: 20 minutes. Time invested weekly: 5 min. Worked ~99% of time. 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 19. 1) Regular Expressions Experiences with Golang in DevOps Problem: - From Jenkins, needed to upload file to another machine. - No Slave option. - No FTP option. - SSL Insecure (due to network restrictions). - No SSH Key admitted, only user/password. - No responsibility from client or other providers. Solution: Golang Script: - Took 1 day. - Generated binaries. - Plugins: - SFTP for Golang - SSH for Golang Worked as expected. Can be used in Windows & Linux Machines. 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 20. 1) Regular Expressions Experiences with Golang in DevOps Problem: - Logs only had a 24 hour margin. - Took to long to recover (around 12 Gb) - Only recovered through FTP. - Usually take around 2 hours to recover everything. - Network inestability through Wi-Fi. - Process was longer through VPN. - Many zipped files. Solution: Golang Script: - Took 2 days. - Generated binaries. - Plugins: - SFTP for Golang - SSH for Golang Process took 30 minutes to recover all the data, only in 1 zip. 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 21. 1) Regular Expressions Experiences with Golang in DevOps Problem: - There was a server migration Project, changing 30 environments from On-Premise to the Cloud (Beta). - Every 2 or 3 days, some of the servers were down. - This affected the delivery times, since some dependencies were loaded from these servers. - The monitor only could be invoked from a custom java Project. - The vendor in charge of the Project didn’t have a way to veify every system. - Every Project had a dependency to another Project (Messaging and SOAP Calls). Solution: Golang Script w/Java Project: - Java project only checked every server and delivered a Json result. - Golang Project took the Json and transformed it to a visual report. - Daily Monitoring Since every day the servers had problems, the report helped to verify the root causes for the 50% of failures. Improved the deliver time for the vendor (a little). 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 22. 1) Regular Expressions Experiences with Golang in DevOps Problem: - With the log obtained from 6, we had to process and check the files for posible failure causes. - No Splunk (team & vendor didn’t have time to invest in learning) - Failures were critical. - Too many log files - Some files were higher tan 1Gb - The logs were a mix of logger registry, Json and XML Solution: Golang Script: - Regular expression to identify the event type. - Normalized the data. - Verified the timestamp for the data (some were in UTC, some in local time) - Report with sorted events. Build time: 2 days and 5 improing iterations. Fetching Data from every log manually: 1-2 days. Fetching Data from every log w/script: 30 min. 2) Ticket fetching 3) Machine restart 4) LDAP & DB Injection 5) File Upload 6) Log Fetcher 7) Weblogic Monitoring 8) Log Analyzer
  • 23. Most of the scripts can be found in my ACN GitHub. In Readme, I’ve included some incredible references. https://github.com/pperez- accenture/ACN-golang-scripts
  • 24. DEMO TIME(If I still have time)
  • 25. NÓS MOVEMOS O MUNDO. Centro Av. Presidente Wilson, 231 29º andar (21) 2240-2030 Cidade Monções Av. Nações Unidas, 11.541 3º andar (11) 4119-0449 Savassi Av. Getúlio Vargas, 671 Sala 800 - 8º andar (31) 3360-8900 Ilha do Leite Rua Sen. José Henrique, 199 2º andar (81) 3018-6299 WWW.CONCRETE.COM.BR
  • Related Search
    We Need Your Support
    Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

    Thanks to everyone for your continued support.

    No, Thanks
    SAVE OUR EARTH

    We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

    More details...

    Sign Now!

    We are very appreciated for your Prompt Action!

    x