Posts

Showing posts from 2024

Leaving Pinterest today

Image
Today's my last day at Pinterest. A journey comes to an end. Hopefully a new one begins also. I'm keeping a few memories from friends and team mates here. Looking back at the journey, the most important points were:  I had the opportunity to mentor 6 promotions in 4 years: Karthik (L5 to L6 MLE, now an L7 MLE Manager) Lu (L5 to L6 to L7) Heng (L4 to L5 to L6) Qingxian (L4 to L5 MLE) ... and there are more that should get promoted this year or the next. I had the opportunity to indirectly influence the growth of my mentees: Jeremy (then L5, now L6 in Security) Deborah (then L4 in ML ranking) Lily (then L4 in Data Analytics) Jason, Chenqi, Anumol, Kanchi, Dinesh, Nishant and many others. I had the opportunity to build and deploy one of the world's largest Flink clusters Set up the infrastructure to support 20 million messages/s. Train and upskill a 15 person team Support 300+ engineers across 24+ teams.  Deliver on the business critical path, on time and without being a bloc

BUCK2 import Maven dependencies

Meta's BUCK2 doesn't yet have Java and Android support (see github issue 394 ), however, it's useful to have a script to autogenerate remote_file and prebuilt_jar rules from Maven coordinates, if the code did work. If you're trying to get BUCK2 to compile Java, save yourself the trouble just yet and use Bazel - it should work for you.  That said, here's a quick attempt at a script to import maven jars for BUCK. If it's useful to you, feel free to use. #!/usr/bin/env bash # # Import a Maven dependency based on the coordinates. # # # Print debug log # set -x # Exit on error set -e if [[ $# != 1 ]]; then echo Usage: $0 com.pinterest.optimus:library:version echo echo Imports the given Maven target into a BUCK file in third_party/ exit 1 fi # Change to third-party directory cd `dirname $0` MVN_COORDS=$1 GROUP_ID=$(echo $MVN_COORDS | cut -f 1 -d:) ARTIFACT_ID=$(echo $MVN_COORDS | cut -f 2 -d:) VERSION=$(echo $MVN_COORDS | cut -f 3 -d:) MAVEN_CENTRAL_UR

How to correctly compile Thrift using Maven

The org.apache.thrift.tools:maven-thrift-plugin is old and largely unmaintained (last updated 2013). It should not be used anymore. The correct plugin to use is org.apache.thrift:thrift-maven-plugin which was last updated in 2017.  The main problem with the maven-thrift-plugin and the thrift-maven-plugin is that it doesn't correctly integrate with the maven generate-sources command (at-least for Maven 3.8.4). The way to fix that is to change the generated sources to be placed in the src/main directory of the maven package. The ` <generator>java</generator> ` is to fix some of the backward incompatible changes on the thrift compiler's command line (usually shows up as an error: " unknown option java:hashcode").  Adding the following to your pom.xml will allow a process where the thrift files can be manually updated with a local command the rest of the compile is straightforward.  <plugin> <!-- Note: trigger rebuild of java classes from thrift